[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:42 dist-test-slave-2j7r krb5kdc[24362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:42 dist-test-slave-2j7r krb5kdc[24362](info): set up 2 sockets
Aug 07 20:12:42 dist-test-slave-2j7r krb5kdc[24362](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 20:12:43.330044 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:12:43 dist-test-slave-2j7r krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:12:43.455224 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:44571
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:34573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:44571
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:43.540609 24385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:43.540838 24385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:43.540861 24385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:43.542028 24385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:43.542083 24385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:43.542098 24385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:43.542112 24385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:43.542125 24385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:43.543607 24385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:44571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:44571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:43.543823 24385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:43.544020 24385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:43.549455 24397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:43.549512 24385 server_base.cc:1034] running on GCE node
W20260807 20:12:43.549489 24395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:43.549440 24399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:43.549901 24385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:43.550199 24385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:43.551370 24385 hybrid_clock.cc:648] HybridClock initialized: now 1786133563551351 us; error 33 us; skew 500 ppm
Aug 07 20:12:43 dist-test-slave-2j7r krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133563, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:43.558326 24385 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:12:43.558810 24385 webserver.cc:460] Webserver started at http://127.23.200.254:35841/ using document root <none> and password file <none>
I20260807 20:12:43.559001 24385 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:43.559042 24385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:43.559140 24385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:43.560340 24385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "36191b5d8d92476d9e7e499c3ffabfee"
format_stamp: "Formatted at 2026-08-07 20:12:43 on dist-test-slave-2j7r"
server_key: "1cb24846a25374b31ed1d7db3e91d76a"
server_key_iv: "7475f7f4e4cc25988a2601f139e23d4e"
server_key_version: "encryptionkey@0"
I20260807 20:12:43.560644 24385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "36191b5d8d92476d9e7e499c3ffabfee"
format_stamp: "Formatted at 2026-08-07 20:12:43 on dist-test-slave-2j7r"
server_key: "1cb24846a25374b31ed1d7db3e91d76a"
server_key_iv: "7475f7f4e4cc25988a2601f139e23d4e"
server_key_version: "encryptionkey@0"
I20260807 20:12:43.561993 24385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:43.562808 24408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:43.562992 24385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:43.563084 24385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "36191b5d8d92476d9e7e499c3ffabfee"
format_stamp: "Formatted at 2026-08-07 20:12:43 on dist-test-slave-2j7r"
server_key: "1cb24846a25374b31ed1d7db3e91d76a"
server_key_iv: "7475f7f4e4cc25988a2601f139e23d4e"
server_key_version: "encryptionkey@0"
I20260807 20:12:43.563143 24385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:43.574086 24385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:43.576049 24385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:43.576181 24385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:43.580145 24385 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:44571
I20260807 20:12:43.580194 24499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:44571 every 8 connection(s)
I20260807 20:12:43.580538 24385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:43.581534 24500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:43.583928 24500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee: Bootstrap starting.
I20260807 20:12:43.584538 24500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:43.584810 24500 log.cc:824] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:43.585605 24500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee: No bootstrap required, opened a new log
I20260807 20:12:43.587225 24500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36191b5d8d92476d9e7e499c3ffabfee" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44571 } }
I20260807 20:12:43.587430 24500 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:43.587467 24500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36191b5d8d92476d9e7e499c3ffabfee, State: Initialized, Role: FOLLOWER
I20260807 20:12:43.587615 24500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36191b5d8d92476d9e7e499c3ffabfee" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44571 } }
I20260807 20:12:43.587682 24500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:43.587714 24500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:43.587772 24500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:43.588305 24500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36191b5d8d92476d9e7e499c3ffabfee" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44571 } }
I20260807 20:12:43.588406 24500 leader_election.cc:302] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36191b5d8d92476d9e7e499c3ffabfee; no voters:
I20260807 20:12:43.588578 24500 leader_election.cc:288] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:43.588613 24506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:43.588773 24506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [term 1 LEADER]: Becoming Leader. State: Replica: 36191b5d8d92476d9e7e499c3ffabfee, State: Running, Role: LEADER
I20260807 20:12:43.588835 24500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:43.588881 24506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36191b5d8d92476d9e7e499c3ffabfee" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44571 } }
I20260807 20:12:43.589355 24508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36191b5d8d92476d9e7e499c3ffabfee. Latest consensus state: current_term: 1 leader_uuid: "36191b5d8d92476d9e7e499c3ffabfee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36191b5d8d92476d9e7e499c3ffabfee" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44571 } } }
I20260807 20:12:43.589443 24508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:43.589519 24507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36191b5d8d92476d9e7e499c3ffabfee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36191b5d8d92476d9e7e499c3ffabfee" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44571 } } }
I20260807 20:12:43.589628 24507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:43.589761 24519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:43.590325 24519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:43.590737 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 24385
I20260807 20:12:43.590852 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:12:43.591647 24355 external_mini_cluster.cc:1428] Setting key 3698626c88795e9934fbfdf114bbfd40
I20260807 20:12:43.592875 24519 catalog_manager.cc:1347] Generated new cluster ID: 87aa89e8932c4a0493c4af6aec2c1b7f
I20260807 20:12:43.592936 24519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:43.599273 24519 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:43.599864 24519 catalog_manager.cc:1504] Loading token signing keys...
Aug 07 20:12:43 dist-test-slave-2j7r krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:43.607486 24519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36191b5d8d92476d9e7e499c3ffabfee: Generated new TSK 0
I20260807 20:12:43.607667 24519 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:43.610384 24532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:43.592530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54468 (local address 127.23.200.254:44571)
0807 20:12:43.592710 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:43.592720 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:43.595518 (+ 2798us) server_negotiation.cc:407] Connection header received
0807 20:12:43.595635 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:43.595642 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:43.595698 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:43.595829 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:43.597777 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:43.599538 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:43.600987 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:43.601640 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:43.601746 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:43.606167 (+ 4421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:43.606172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:43.606172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:43.606185 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:43.608417 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:43.608903 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:43.608907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:43.608907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:43.609024 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:43.609331 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:43.609332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:43.609332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:43.609618 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:43.609778 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:43.609978 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 20:12:43.610131 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:12:43.641119 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:44571
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:34573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:43.730630 24539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:43.730834 24539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:43.730858 24539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:43.732017 24539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:43.732062 24539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:43.732091 24539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:12:43.733639 24539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:44571
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:43.733878 24539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:43.734091 24539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:43.739075 24554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:43.739106 24548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:43.739127 24547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:43.739217 24539 server_base.cc:1034] running on GCE node
I20260807 20:12:43.739531 24539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:43.739755 24539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:43.740899 24539 hybrid_clock.cc:648] HybridClock initialized: now 1786133563740884 us; error 27 us; skew 500 ppm
Aug 07 20:12:43 dist-test-slave-2j7r krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133563, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:43.743394 24539 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:12:43.743824 24539 webserver.cc:460] Webserver started at http://127.23.200.193:40637/ using document root <none> and password file <none>
I20260807 20:12:43.744029 24539 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:43.744076 24539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:43.744195 24539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:43.745245 24539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "b12c0d37899e4fd486c169801a407d50"
format_stamp: "Formatted at 2026-08-07 20:12:43 on dist-test-slave-2j7r"
server_key: "1eb93acb0514b5b242bca69a89551332"
server_key_iv: "7467a521fb34b233aff3bc6ffaa5797d"
server_key_version: "encryptionkey@0"
I20260807 20:12:43.745607 24539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "b12c0d37899e4fd486c169801a407d50"
format_stamp: "Formatted at 2026-08-07 20:12:43 on dist-test-slave-2j7r"
server_key: "1eb93acb0514b5b242bca69a89551332"
server_key_iv: "7467a521fb34b233aff3bc6ffaa5797d"
server_key_version: "encryptionkey@0"
I20260807 20:12:43.746845 24539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:43.747496 24564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:43.747641 24539 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:43.747726 24539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "b12c0d37899e4fd486c169801a407d50"
format_stamp: "Formatted at 2026-08-07 20:12:43 on dist-test-slave-2j7r"
server_key: "1eb93acb0514b5b242bca69a89551332"
server_key_iv: "7467a521fb34b233aff3bc6ffaa5797d"
server_key_version: "encryptionkey@0"
I20260807 20:12:43.747785 24539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:43.763381 24539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:43.765571 24539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:43.765714 24539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:43.765961 24539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:43.766310 24539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:43.766346 24539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:43.766367 24539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:43.766378 24539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:43.772042 24539 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:40567
I20260807 20:12:43.772089 24755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:40567 every 8 connection(s)
I20260807 20:12:43.772400 24539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:12:43.777266 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 24539
I20260807 20:12:43.777348 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:12:43.777516 24355 external_mini_cluster.cc:1428] Setting key 349310e12f3e9f9868968cb0a37f3918
Aug 07 20:12:43 dist-test-slave-2j7r krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133563, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:43.781877 24532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:43.773136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:47179 (local address 127.23.200.254:44571)
0807 20:12:43.773189 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:43.773191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:43.773408 (+ 217us) server_negotiation.cc:407] Connection header received
0807 20:12:43.773921 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:43.773922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:43.773939 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:43.773989 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:43.775147 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:43.775648 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:43.776623 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:43.776828 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:43.776885 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:43.779152 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:43.779153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:43.779154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:43.779162 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:43.780503 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:43.780873 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:43.780875 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:43.780875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:43.780937 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:43.781204 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:43.781205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:43.781205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:43.781324 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:43.781404 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:43.781682 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 20:12:43.781783 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 20:12:43.782016 24759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:43.773104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:44571 (local address 127.23.200.193:47179)
0807 20:12:43.773288 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:43.773325 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:43.773822 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:43.774072 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:43.774076 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:43.774853 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:43.775047 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:43.775052 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:43.775733 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:43.775734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:43.776538 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:43.776541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:43.776898 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:43.776899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:43.777022 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:43.777547 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:43.777559 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:43.779069 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:43.780583 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:43.780590 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:43.780593 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:43.780806 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:43.780990 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:43.780990 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:43.780991 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:43.781144 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:43.781395 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:43.781402 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:43.781549 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 20:12:43.781634 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 20:12:43.781795 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":80,"threads_started":1}
I20260807 20:12:43.782497 24756 heartbeater.cc:347] Connected to a master server at 127.23.200.254:44571
I20260807 20:12:43.782609 24756 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:43.782811 24756 heartbeater.cc:511] Master 127.23.200.254:44571 requested a full tablet report, sending...
I20260807 20:12:43.783406 24443 ts_manager.cc:194] Registered new tserver with Master: b12c0d37899e4fd486c169801a407d50 (127.23.200.193:40567)
I20260807 20:12:43.784027 24443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:47179
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:12:43.804504 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:44571
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:34573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:43.887341 24765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:43.887547 24765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:43.887569 24765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:43.888727 24765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:43.888774 24765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:43.888801 24765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:12:43.890412 24765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:44571
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:43.890658 24765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:43.890890 24765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:43.896023 24777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:43.896061 24776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:43.896222 24765 server_base.cc:1034] running on GCE node
W20260807 20:12:43.896061 24782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:43.896525 24765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:43.896745 24765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:43.897900 24765 hybrid_clock.cc:648] HybridClock initialized: now 1786133563897883 us; error 30 us; skew 500 ppm
Aug 07 20:12:43 dist-test-slave-2j7r krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133563, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:43.900442 24765 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:12:43.900841 24765 webserver.cc:460] Webserver started at http://127.23.200.194:35007/ using document root <none> and password file <none>
I20260807 20:12:43.901046 24765 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:43.901101 24765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:43.901216 24765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:43.902225 24765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "c1a654d320444563afa4c89078027caf"
format_stamp: "Formatted at 2026-08-07 20:12:43 on dist-test-slave-2j7r"
server_key: "f7417726b6d5072109bf4b12f558044a"
server_key_iv: "f111b2b0796552f82f95609449dcd74a"
server_key_version: "encryptionkey@0"
I20260807 20:12:43.902527 24765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "c1a654d320444563afa4c89078027caf"
format_stamp: "Formatted at 2026-08-07 20:12:43 on dist-test-slave-2j7r"
server_key: "f7417726b6d5072109bf4b12f558044a"
server_key_iv: "f111b2b0796552f82f95609449dcd74a"
server_key_version: "encryptionkey@0"
I20260807 20:12:43.903703 24765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 20:12:43.904325 24794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:43.904477 24765 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:43.904565 24765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "c1a654d320444563afa4c89078027caf"
format_stamp: "Formatted at 2026-08-07 20:12:43 on dist-test-slave-2j7r"
server_key: "f7417726b6d5072109bf4b12f558044a"
server_key_iv: "f111b2b0796552f82f95609449dcd74a"
server_key_version: "encryptionkey@0"
I20260807 20:12:43.904624 24765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:43.917840 24765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:43.919931 24765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:43.920069 24765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:43.920295 24765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:43.920593 24765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:43.920625 24765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:43.920653 24765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:43.920677 24765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:43.926704 24765 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:37231
I20260807 20:12:43.926781 24964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:37231 every 8 connection(s)
I20260807 20:12:43.927068 24765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:12:43.929780 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 24765
I20260807 20:12:43.929867 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:12:43.930023 24355 external_mini_cluster.cc:1428] Setting key dd6b5d0c9cff2d0b23956138df722e60
Aug 07 20:12:43 dist-test-slave-2j7r krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133563, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:43.937003 24532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:43.927913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:52383 (local address 127.23.200.254:44571)
0807 20:12:43.927971 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:43.927973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:43.928211 (+ 238us) server_negotiation.cc:407] Connection header received
0807 20:12:43.928649 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:43.928650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:43.928662 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:43.928723 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:43.929740 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:43.930362 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:43.931455 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:43.931705 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:43.931777 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:43.934194 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:43.934196 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:43.934196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:43.934205 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:43.935534 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:43.936009 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:43.936010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:43.936010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:43.936064 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:43.936341 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:43.936342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:43.936342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:43.936478 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:43.936531 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:43.936862 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 20:12:43.936932 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 20:12:43.937305 24973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:43.927882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:44571 (local address 127.23.200.194:52383)
0807 20:12:43.928057 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:43.928101 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:43.928570 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:43.928786 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:43.928790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:43.929416 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:43.929637 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:43.929645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:43.930449 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:43.930450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:43.931341 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:43.931344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:43.931777 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:43.931778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:43.931876 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:43.932428 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:43.932440 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:43.934082 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:43.935623 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:43.935630 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:43.935633 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:43.935934 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:43.936125 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:43.936126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:43.936126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:43.936270 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:43.936540 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:43.936546 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:43.936726 (+ 180us) client_negotiation.cc:769] Sending connection context
0807 20:12:43.936851 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 20:12:43.937030 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260807 20:12:43.937888 24965 heartbeater.cc:347] Connected to a master server at 127.23.200.254:44571
I20260807 20:12:43.938004 24965 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:43.938220 24965 heartbeater.cc:511] Master 127.23.200.254:44571 requested a full tablet report, sending...
I20260807 20:12:43.938647 24443 ts_manager.cc:194] Registered new tserver with Master: c1a654d320444563afa4c89078027caf (127.23.200.194:37231)
I20260807 20:12:43.939024 24443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:52383
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:12:43.960569 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:44571
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:34573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:44.044250 24976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:44.044461 24976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:44.044484 24976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:44.045660 24976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:44.045701 24976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:44.045727 24976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:12:44.047175 24976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:44571
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:44.047408 24976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:44.047624 24976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:44.052737 24984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:44.052769 24985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:44.052795 24976 server_base.cc:1034] running on GCE node
W20260807 20:12:44.052757 24987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:44.053203 24976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:44.053582 24976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:44.054759 24976 hybrid_clock.cc:648] HybridClock initialized: now 1786133564054741 us; error 25 us; skew 500 ppm
Aug 07 20:12:44 dist-test-slave-2j7r krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133564, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:44.057415 24976 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:12:44.057837 24976 webserver.cc:460] Webserver started at http://127.23.200.195:42433/ using document root <none> and password file <none>
I20260807 20:12:44.058022 24976 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:44.058061 24976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:44.058157 24976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:44.059170 24976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "c5e77401e4a94b39b63b19c36866e517"
format_stamp: "Formatted at 2026-08-07 20:12:44 on dist-test-slave-2j7r"
server_key: "2130aa06e825a6ffad80871b66b96960"
server_key_iv: "285558156777ecde7fb062d32f7af326"
server_key_version: "encryptionkey@0"
I20260807 20:12:44.059490 24976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "c5e77401e4a94b39b63b19c36866e517"
format_stamp: "Formatted at 2026-08-07 20:12:44 on dist-test-slave-2j7r"
server_key: "2130aa06e825a6ffad80871b66b96960"
server_key_iv: "285558156777ecde7fb062d32f7af326"
server_key_version: "encryptionkey@0"
I20260807 20:12:44.060703 24976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:12:44.061576 24995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:44.061750 24976 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:12:44.061839 24976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "c5e77401e4a94b39b63b19c36866e517"
format_stamp: "Formatted at 2026-08-07 20:12:44 on dist-test-slave-2j7r"
server_key: "2130aa06e825a6ffad80871b66b96960"
server_key_iv: "285558156777ecde7fb062d32f7af326"
server_key_version: "encryptionkey@0"
I20260807 20:12:44.061895 24976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:44.071450 24976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:44.073496 24976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:44.073611 24976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:44.073786 24976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:44.074074 24976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:44.074105 24976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:44.074124 24976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:44.074136 24976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:44.079789 24976 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:43563
I20260807 20:12:44.079854 25172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:43563 every 8 connection(s)
I20260807 20:12:44.080145 24976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:12:44.086270 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 24976
Aug 07 20:12:44 dist-test-slave-2j7r krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133564, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:44.086350 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:12:44.086546 24355 external_mini_cluster.cc:1428] Setting key 0b1a802cc20f8cd587aaad314c93434a
I20260807 20:12:44.089665 24532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:44.081104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:39401 (local address 127.23.200.254:44571)
0807 20:12:44.081155 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:44.081157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:44.081379 (+ 222us) server_negotiation.cc:407] Connection header received
0807 20:12:44.081886 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:44.081887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:44.081908 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:44.081964 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:44.082930 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:44.083426 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:44.084393 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:44.084582 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:44.084638 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:44.086862 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:44.086864 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:44.086865 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:44.086874 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:44.088282 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:44.088679 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:44.088680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:44.088681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:44.088742 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:44.088961 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:44.088961 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:44.088961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:44.089145 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:44.089197 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:44.089454 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 20:12:44.089572 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:12:44.089804 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:44.081070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:44571 (local address 127.23.200.195:39401)
0807 20:12:44.081256 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:44.081294 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:44.081788 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:44.082046 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:44.082051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:44.082620 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:44.082818 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:44.082823 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:44.083513 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:44.083514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:44.084308 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:44.084310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:44.084650 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:44.084651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:44.084778 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:44.085275 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:44.085284 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:44.086738 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:44.088361 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:44.088368 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:44.088370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:44.088599 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:44.088803 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:44.088804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:44.088804 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:44.088916 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:44.089190 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:44.089196 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:44.089314 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 20:12:44.089407 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 20:12:44.089570 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260807 20:12:44.090286 25173 heartbeater.cc:347] Connected to a master server at 127.23.200.254:44571
I20260807 20:12:44.090386 25173 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:44.090576 25173 heartbeater.cc:511] Master 127.23.200.254:44571 requested a full tablet report, sending...
I20260807 20:12:44.090938 24443 ts_manager.cc:194] Registered new tserver with Master: c5e77401e4a94b39b63b19c36866e517 (127.23.200.195:43563)
I20260807 20:12:44.091281 24443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:39401
I20260807 20:12:44.098804 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 20:12:44 dist-test-slave-2j7r krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133564, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 20:12:44 dist-test-slave-2j7r krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133564, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.200.193@KRBTEST.COM
I20260807 20:12:44.123076 25184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:44.115530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57210 (local address 127.23.200.193:40567)
0807 20:12:44.115699 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:44.115706 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:44.115721 (+ 15us) server_negotiation.cc:407] Connection header received
0807 20:12:44.115750 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:44.115753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:44.115768 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:44.115843 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:44.116335 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:44.117121 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:44.117824 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:44.118178 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:44.118242 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:44.120336 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:44.120344 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:44.120345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:44.120353 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:44.121927 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:44.122333 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:44.122337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:44.122337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:44.122426 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:44.122591 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:44.122591 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:44.122592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:44.122738 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:44.122856 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:44.122895 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 20:12:44.122970 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
W20260807 20:12:44.123545 24615 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57210
Aug 07 20:12:44 dist-test-slave-2j7r krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 20:12:44 dist-test-slave-2j7r krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.193@KRBTEST.COM
I20260807 20:12:44.146950 25184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:44.140132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57218 (local address 127.23.200.193:40567)
0807 20:12:44.140197 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:44.140199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:44.140238 (+ 39us) server_negotiation.cc:407] Connection header received
0807 20:12:44.140307 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:44.140308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:44.140320 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:44.140365 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:44.140951 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:44.141490 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:44.142195 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:44.142402 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:44.142457 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:44.144578 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:44.144579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:44.144579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:44.144590 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:44.145946 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:44.146252 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:44.146253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:44.146254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:44.146306 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:44.146472 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:44.146473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:44.146473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:44.146566 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:44.146672 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:44.146755 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 20:12:44.146833 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 20:12:44.147492 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 24539
I20260807 20:12:44.168987 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 24765
I20260807 20:12:44.188936 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 24976
I20260807 20:12:44.196061 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 24385
2026-08-07T20:12:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2133 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:44 dist-test-slave-2j7r krb5kdc[25193](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:44 dist-test-slave-2j7r krb5kdc[25193](info): set up 2 sockets
Aug 07 20:12:44 dist-test-slave-2j7r krb5kdc[25193](info): commencing operation
krb5kdc: starting...
W20260807 20:12:45.361191 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:12:45 dist-test-slave-2j7r krb5kdc[25193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:12:45.484951 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:34887
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:34553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:34887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:45.568611 25213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:45.568818 25213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:45.568847 25213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:45.570025 25213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:45.570067 25213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:45.570086 25213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:45.570102 25213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:45.570116 25213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:45.571602 25213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:34887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:34887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:45.571867 25213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:45.572082 25213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:12:45.576872 25213 server_base.cc:1034] running on GCE node
W20260807 20:12:45.576900 25220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:45.576884 25224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:45.577015 25221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:45.577373 25213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:45.577646 25213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:45.578796 25213 hybrid_clock.cc:648] HybridClock initialized: now 1786133565578771 us; error 32 us; skew 500 ppm
Aug 07 20:12:45 dist-test-slave-2j7r krb5kdc[25193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133565, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:45.581395 25213 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:12:45.581844 25213 webserver.cc:460] Webserver started at http://127.23.200.254:36357/ using document root <none> and password file <none>
I20260807 20:12:45.582072 25213 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:45.582119 25213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:45.582244 25213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:45.583318 25213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "519a1cecfdd4466b87732172da0627f7"
format_stamp: "Formatted at 2026-08-07 20:12:45 on dist-test-slave-2j7r"
server_key: "bb358b50f9b39b12f562b5b1c5bc1c77"
server_key_iv: "ab1b5f94ef81153b00df93e8f8db1052"
server_key_version: "encryptionkey@0"
I20260807 20:12:45.583649 25213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "519a1cecfdd4466b87732172da0627f7"
format_stamp: "Formatted at 2026-08-07 20:12:45 on dist-test-slave-2j7r"
server_key: "bb358b50f9b39b12f562b5b1c5bc1c77"
server_key_iv: "ab1b5f94ef81153b00df93e8f8db1052"
server_key_version: "encryptionkey@0"
I20260807 20:12:45.584903 25213 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:45.585784 25233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:45.586098 25213 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:45.586191 25213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "519a1cecfdd4466b87732172da0627f7"
format_stamp: "Formatted at 2026-08-07 20:12:45 on dist-test-slave-2j7r"
server_key: "bb358b50f9b39b12f562b5b1c5bc1c77"
server_key_iv: "ab1b5f94ef81153b00df93e8f8db1052"
server_key_version: "encryptionkey@0"
I20260807 20:12:45.586256 25213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:45.626132 25213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:45.628139 25213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:45.628285 25213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:45.632253 25213 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:34887
I20260807 20:12:45.632294 25309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:34887 every 8 connection(s)
I20260807 20:12:45.632638 25213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:45.633594 25311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:45.635881 25311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7: Bootstrap starting.
I20260807 20:12:45.636421 25311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:45.636678 25311 log.cc:824] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:45.637344 25311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7: No bootstrap required, opened a new log
I20260807 20:12:45.638880 25311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519a1cecfdd4466b87732172da0627f7" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 34887 } }
I20260807 20:12:45.639019 25311 raft_consensus.cc:374] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:45.639041 25311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 519a1cecfdd4466b87732172da0627f7, State: Initialized, Role: FOLLOWER
I20260807 20:12:45.639154 25311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519a1cecfdd4466b87732172da0627f7" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 34887 } }
I20260807 20:12:45.639235 25311 raft_consensus.cc:388] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:45.639266 25311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:45.639300 25311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:45.639819 25311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519a1cecfdd4466b87732172da0627f7" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 34887 } }
I20260807 20:12:45.639907 25311 leader_election.cc:302] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 519a1cecfdd4466b87732172da0627f7; no voters:
I20260807 20:12:45.640033 25311 leader_election.cc:288] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:45.640079 25317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:45.640257 25317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [term 1 LEADER]: Becoming Leader. State: Replica: 519a1cecfdd4466b87732172da0627f7, State: Running, Role: LEADER
I20260807 20:12:45.640264 25311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:45.640359 25317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519a1cecfdd4466b87732172da0627f7" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 34887 } }
I20260807 20:12:45.640717 25319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 519a1cecfdd4466b87732172da0627f7. Latest consensus state: current_term: 1 leader_uuid: "519a1cecfdd4466b87732172da0627f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519a1cecfdd4466b87732172da0627f7" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 34887 } } }
I20260807 20:12:45.640796 25319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:45.641036 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 25213
I20260807 20:12:45.641125 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:12:45.641285 24355 external_mini_cluster.cc:1428] Setting key 911fa17ad399b138df489f9bef96365d
I20260807 20:12:45.641412 25329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:45.641767 25318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "519a1cecfdd4466b87732172da0627f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "519a1cecfdd4466b87732172da0627f7" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 34887 } } }
I20260807 20:12:45.641902 25318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:45.641952 25329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:45.643575 25329 catalog_manager.cc:1347] Generated new cluster ID: f2b5e064ac594648a05ff98b4ca14894
I20260807 20:12:45.643627 25329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:12:45 dist-test-slave-2j7r krb5kdc[25193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:45.652915 25340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:45.641857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59332 (local address 127.23.200.254:34887)
0807 20:12:45.642284 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:45.642294 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:45.642324 (+ 30us) server_negotiation.cc:407] Connection header received
0807 20:12:45.642556 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:45.642564 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:45.642624 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:45.642770 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:45.643454 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:45.644840 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:45.645960 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:45.646564 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:45.646655 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:45.649355 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:45.649360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:45.649360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:45.649379 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:45.651217 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:45.651579 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:45.651583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:45.651583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:45.651688 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:45.651858 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:45.651859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:45.651860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:45.652175 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:45.652327 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:45.652416 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 20:12:45.652664 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":74,"threads_started":1}
I20260807 20:12:45.659562 25329 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:45.660079 25329 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:45.665529 25329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 519a1cecfdd4466b87732172da0627f7: Generated new TSK 0
I20260807 20:12:45.665694 25329 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:12:45.697152 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:34887
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:34553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:45.791414 25350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:45.791635 25350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:45.791661 25350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:45.792829 25350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:45.792874 25350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:45.792902 25350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:12:45.794478 25350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:34887
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:45.794682 25350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:45.794883 25350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:45.799883 25361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:45.799898 25366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:45.800006 25350 server_base.cc:1034] running on GCE node
W20260807 20:12:45.799880 25359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:45.800320 25350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:45.800597 25350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:45.801744 25350 hybrid_clock.cc:648] HybridClock initialized: now 1786133565801730 us; error 28 us; skew 500 ppm
Aug 07 20:12:45 dist-test-slave-2j7r krb5kdc[25193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133565, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:45.804255 25350 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:12:45.804666 25350 webserver.cc:460] Webserver started at http://127.23.200.193:40741/ using document root <none> and password file <none>
I20260807 20:12:45.804858 25350 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:45.804905 25350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:45.804998 25350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:45.806052 25350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "1ba3fbf634724c89a294fcbc40d5f723"
format_stamp: "Formatted at 2026-08-07 20:12:45 on dist-test-slave-2j7r"
server_key: "399f5c48d53c1530ba779cd8feed8d59"
server_key_iv: "424149c8e401ca60525933f9e29a6b8a"
server_key_version: "encryptionkey@0"
I20260807 20:12:45.806376 25350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "1ba3fbf634724c89a294fcbc40d5f723"
format_stamp: "Formatted at 2026-08-07 20:12:45 on dist-test-slave-2j7r"
server_key: "399f5c48d53c1530ba779cd8feed8d59"
server_key_iv: "424149c8e401ca60525933f9e29a6b8a"
server_key_version: "encryptionkey@0"
I20260807 20:12:45.807646 25350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:12:45.808306 25376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:45.808450 25350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:45.808537 25350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "1ba3fbf634724c89a294fcbc40d5f723"
format_stamp: "Formatted at 2026-08-07 20:12:45 on dist-test-slave-2j7r"
server_key: "399f5c48d53c1530ba779cd8feed8d59"
server_key_iv: "424149c8e401ca60525933f9e29a6b8a"
server_key_version: "encryptionkey@0"
I20260807 20:12:45.808590 25350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:45.819790 25350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:45.821602 25350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:45.821713 25350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:45.821908 25350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:45.822172 25350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:45.822203 25350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:45.822224 25350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:45.822235 25350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:45.827517 25350 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:35229
I20260807 20:12:45.827587 25538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:35229 every 8 connection(s)
I20260807 20:12:45.827857 25350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:12:45.833163 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 25350
I20260807 20:12:45.833259 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:12:45.833441 24355 external_mini_cluster.cc:1428] Setting key 13b57662ff163f1a905db6f2d4c7a773
Aug 07 20:12:45 dist-test-slave-2j7r krb5kdc[25193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133565, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:45.837138 25340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:45.828560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:50811 (local address 127.23.200.254:34887)
0807 20:12:45.828627 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:45.828629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:45.828789 (+ 160us) server_negotiation.cc:407] Connection header received
0807 20:12:45.829860 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:45.829862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:45.829879 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:45.829932 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:45.830563 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:45.831070 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:45.832016 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:45.832220 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:45.832275 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:45.834637 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:45.834638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:45.834639 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:45.834646 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:45.835880 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:45.836245 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:45.836246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:45.836247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:45.836309 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:45.836505 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:45.836506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:45.836507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:45.836638 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:45.836697 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:45.836942 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 20:12:45.837038 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 20:12:45.837271 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:45.828527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:34887 (local address 127.23.200.193:50811)
0807 20:12:45.828674 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:45.828711 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:45.829749 (+ 1038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:45.829999 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:45.830003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:45.830254 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:45.830472 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:45.830477 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:45.831150 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:45.831151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:45.831937 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:45.831939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:45.832288 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:45.832289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:45.832403 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:45.832836 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:45.832845 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:45.834553 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:45.835961 (+ 1408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:45.835968 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:45.835970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:45.836182 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:45.836364 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:45.836364 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:45.836364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:45.836455 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:45.836694 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:45.836697 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:45.836808 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 20:12:45.836905 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 20:12:45.837028 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260807 20:12:45.837764 25542 heartbeater.cc:347] Connected to a master server at 127.23.200.254:34887
I20260807 20:12:45.837857 25542 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:45.838049 25542 heartbeater.cc:511] Master 127.23.200.254:34887 requested a full tablet report, sending...
I20260807 20:12:45.838551 25260 ts_manager.cc:194] Registered new tserver with Master: 1ba3fbf634724c89a294fcbc40d5f723 (127.23.200.193:35229)
I20260807 20:12:45.839152 25260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:50811
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:12:45.860531 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:34887
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:34553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:45.943825 25549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:45.944048 25549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:45.944075 25549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:45.945271 25549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:45.945317 25549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:45.945350 25549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:12:45.947332 25549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:34887
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:45.947584 25549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:45.947795 25549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:45.953594 25565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:45.953588 25549 server_base.cc:1034] running on GCE node
W20260807 20:12:45.953576 25556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:45.953562 25558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:45.954092 25549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:45.954353 25549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:45.955579 25549 hybrid_clock.cc:648] HybridClock initialized: now 1786133565955554 us; error 34 us; skew 500 ppm
Aug 07 20:12:45 dist-test-slave-2j7r krb5kdc[25193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133565, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:45.958280 25549 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:12:45.958884 25549 webserver.cc:460] Webserver started at http://127.23.200.194:35115/ using document root <none> and password file <none>
I20260807 20:12:45.959167 25549 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:45.959223 25549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:45.959342 25549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:45.960458 25549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "80fed330578d45a2af04af0e5266a270"
format_stamp: "Formatted at 2026-08-07 20:12:45 on dist-test-slave-2j7r"
server_key: "1e5e9e3c629069f4ad6d224d53619f66"
server_key_iv: "df5cfc34f8c75e1869d24df69cb17a93"
server_key_version: "encryptionkey@0"
I20260807 20:12:45.960790 25549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "80fed330578d45a2af04af0e5266a270"
format_stamp: "Formatted at 2026-08-07 20:12:45 on dist-test-slave-2j7r"
server_key: "1e5e9e3c629069f4ad6d224d53619f66"
server_key_iv: "df5cfc34f8c75e1869d24df69cb17a93"
server_key_version: "encryptionkey@0"
I20260807 20:12:45.962275 25549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:12:45.963117 25572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:45.963340 25549 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:45.963434 25549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "80fed330578d45a2af04af0e5266a270"
format_stamp: "Formatted at 2026-08-07 20:12:45 on dist-test-slave-2j7r"
server_key: "1e5e9e3c629069f4ad6d224d53619f66"
server_key_iv: "df5cfc34f8c75e1869d24df69cb17a93"
server_key_version: "encryptionkey@0"
I20260807 20:12:45.963516 25549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:45.992491 25549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:45.995042 25549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:45.995195 25549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:45.995451 25549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:45.995757 25549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:45.995790 25549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:45.995820 25549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:45.995844 25549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:46.003732 25549 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:44385
I20260807 20:12:46.003805 25746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:44385 every 8 connection(s)
I20260807 20:12:46.004140 25549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:12:46.006650 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 25549
I20260807 20:12:46.006724 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:12:46.006875 24355 external_mini_cluster.cc:1428] Setting key 3474b41648ba43de87470867794bb54c
Aug 07 20:12:46 dist-test-slave-2j7r krb5kdc[25193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133565, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:46.014338 25340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.004907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:39155 (local address 127.23.200.254:34887)
0807 20:12:46.004957 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:46.004959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:46.005188 (+ 229us) server_negotiation.cc:407] Connection header received
0807 20:12:46.006170 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:46.006171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:46.006182 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:46.006244 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:46.007003 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.007637 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.008731 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.008925 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.008982 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.011182 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:46.011183 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:46.011184 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:46.011193 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:46.012772 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:46.013309 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:46.013310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:46.013311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:46.013374 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:46.013662 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:46.013663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:46.013663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:46.013826 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:46.013902 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:46.014156 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 20:12:46.014243 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:12:46.014410 25749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.004881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:34887 (local address 127.23.200.194:39155)
0807 20:12:46.005049 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:46.005116 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:46.006065 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:46.006369 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:46.006374 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:46.006643 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:46.006908 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:46.006915 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.007762 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.007763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:46.008640 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:46.008643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.008996 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.008997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:46.009161 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.009660 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:46.009670 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:46.011085 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:46.012874 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:46.012882 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:46.012884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:46.013181 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:46.013469 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:46.013470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:46.013471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:46.013583 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:46.013883 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:46.013887 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:46.014023 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 20:12:46.014099 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 20:12:46.014227 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
I20260807 20:12:46.015055 25747 heartbeater.cc:347] Connected to a master server at 127.23.200.254:34887
I20260807 20:12:46.015156 25747 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:46.015332 25747 heartbeater.cc:511] Master 127.23.200.254:34887 requested a full tablet report, sending...
I20260807 20:12:46.015750 25260 ts_manager.cc:194] Registered new tserver with Master: 80fed330578d45a2af04af0e5266a270 (127.23.200.194:44385)
I20260807 20:12:46.016250 25260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:39155
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:12:46.035599 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:34887
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:34553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:46.118184 25756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:46.118398 25756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:46.118431 25756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:46.119607 25756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:46.119649 25756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:46.119684 25756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:12:46.121276 25756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:34887
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:46.121523 25756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:46.121738 25756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:12:46.126866 25756 server_base.cc:1034] running on GCE node
W20260807 20:12:46.126840 25767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:46.126878 25770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:46.126849 25766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:46.127332 25756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:46.127612 25756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:46.128750 25756 hybrid_clock.cc:648] HybridClock initialized: now 1786133566128734 us; error 33 us; skew 500 ppm
Aug 07 20:12:46 dist-test-slave-2j7r krb5kdc[25193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133566, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:46.131223 25756 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:12:46.131714 25756 webserver.cc:460] Webserver started at http://127.23.200.195:42533/ using document root <none> and password file <none>
I20260807 20:12:46.131937 25756 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:46.131986 25756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:46.132114 25756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:46.133203 25756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "6f78f3e7448c4e99afdce3d0c7df250c"
format_stamp: "Formatted at 2026-08-07 20:12:46 on dist-test-slave-2j7r"
server_key: "d297c8bca842fe2433fba0d14b9a1a47"
server_key_iv: "f89738f5b3c51421d433a195bbfa859f"
server_key_version: "encryptionkey@0"
I20260807 20:12:46.133555 25756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "6f78f3e7448c4e99afdce3d0c7df250c"
format_stamp: "Formatted at 2026-08-07 20:12:46 on dist-test-slave-2j7r"
server_key: "d297c8bca842fe2433fba0d14b9a1a47"
server_key_iv: "f89738f5b3c51421d433a195bbfa859f"
server_key_version: "encryptionkey@0"
I20260807 20:12:46.134814 25756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:46.135524 25780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:46.135676 25756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:46.135751 25756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "6f78f3e7448c4e99afdce3d0c7df250c"
format_stamp: "Formatted at 2026-08-07 20:12:46 on dist-test-slave-2j7r"
server_key: "d297c8bca842fe2433fba0d14b9a1a47"
server_key_iv: "f89738f5b3c51421d433a195bbfa859f"
server_key_version: "encryptionkey@0"
I20260807 20:12:46.135811 25756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:46.155869 25756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:46.157773 25756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:46.157883 25756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:46.158075 25756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:46.158330 25756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:46.158358 25756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:46.158380 25756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:46.158389 25756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:46.163829 25756 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:40627
I20260807 20:12:46.163913 25948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:40627 every 8 connection(s)
I20260807 20:12:46.164199 25756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
Aug 07 20:12:46 dist-test-slave-2j7r krb5kdc[25193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133566, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:46.171015 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 25756
I20260807 20:12:46.171106 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:12:46.171275 24355 external_mini_cluster.cc:1428] Setting key f8bde2968268d40e19d18afb61b0306d
I20260807 20:12:46.174016 25340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.164977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:52029 (local address 127.23.200.254:34887)
0807 20:12:46.165034 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:46.165036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:46.165372 (+ 336us) server_negotiation.cc:407] Connection header received
0807 20:12:46.166482 (+ 1110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:46.166484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:46.166502 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:46.166554 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:46.167203 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.167774 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.168848 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.169044 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.169134 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.171446 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:46.171449 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:46.171449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:46.171457 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:46.172756 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:46.173159 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:46.173159 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:46.173160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:46.173208 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:46.173462 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:46.173463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:46.173464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:46.173584 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:46.173647 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:46.173887 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 20:12:46.173953 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:12:46.174185 25951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.164952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:34887 (local address 127.23.200.195:52029)
0807 20:12:46.165151 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:46.165193 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:46.166291 (+ 1098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:46.166637 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:46.166640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:46.166896 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:46.167116 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:46.167121 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.167887 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.167888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:46.168740 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:46.168742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.169161 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.169162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:46.169270 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.169738 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:46.169748 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:46.171314 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:46.172822 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:46.172829 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:46.172830 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:46.173066 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:46.173254 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:46.173254 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:46.173255 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:46.173351 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:46.173643 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:46.173647 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:46.173760 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 20:12:46.173848 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 20:12:46.173990 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":77,"threads_started":1}
I20260807 20:12:46.174659 25949 heartbeater.cc:347] Connected to a master server at 127.23.200.254:34887
I20260807 20:12:46.174754 25949 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:46.174938 25949 heartbeater.cc:511] Master 127.23.200.254:34887 requested a full tablet report, sending...
I20260807 20:12:46.175297 25260 ts_manager.cc:194] Registered new tserver with Master: 6f78f3e7448c4e99afdce3d0c7df250c (127.23.200.195:40627)
I20260807 20:12:46.175649 25260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:52029
I20260807 20:12:46.183076 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:12:46.189960 25340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.184574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59336 (local address 127.23.200.254:34887)
0807 20:12:46.184631 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:46.184633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:46.184709 (+ 76us) server_negotiation.cc:407] Connection header received
0807 20:12:46.184751 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:46.184752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:46.184761 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:46.184809 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:46.185350 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.185809 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.186522 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.186716 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.186777 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.187827 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:46.187828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:46.187829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:46.187836 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:46.188952 (+ 1116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:46.189281 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:46.189282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:46.189282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:46.189329 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:46.189489 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:46.189489 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:46.189490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:46.189617 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:46.189686 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:46.189839 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 20:12:46.189899 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:12:46.192006 25260 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:12:46.193656 25260 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:12:46.200142 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.195948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:40627 (local address 127.0.0.1:47932)
0807 20:12:46.196288 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:46.196298 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:46.196362 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:46.196541 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:46.196545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:46.196547 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:46.196631 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:46.196694 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.198145 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.198146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:46.199189 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:46.199191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.199837 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.199839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:46.199925 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.199938 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 20:12:46.200005 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 20:12:46.200059 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":49,"threads_started":1}
I20260807 20:12:46.200225 25971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.196131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:35229 (local address 127.0.0.1:50758)
0807 20:12:46.196591 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:46.196601 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:46.196659 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:46.196908 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:46.196909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:46.196911 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:46.196955 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:46.196957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.198243 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.198244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:46.199097 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:46.199100 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.199990 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.199991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:46.200043 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.200046 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:12:46.200093 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 20:12:46.200124 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":69,"threads_started":1}
I20260807 20:12:46.200227 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.196357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50758 (local address 127.23.200.193:35229)
0807 20:12:46.196606 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:46.196611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:46.196648 (+ 37us) server_negotiation.cc:407] Connection header received
0807 20:12:46.196789 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:46.196792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:46.196809 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:46.197028 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:46.197132 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.198151 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.199172 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.199905 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.199986 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.200043 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:46.200106 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 20:12:46.200148 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":92,"threads_started":1}
I20260807 20:12:46.200393 25969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.195948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:44385 (local address 127.0.0.1:36878)
0807 20:12:46.196114 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:46.196160 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:46.196230 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:46.196955 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:46.196956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:46.196958 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:46.197006 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:46.197008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.197904 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.197905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:46.199157 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:46.199160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.200015 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.200016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:46.200083 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.200087 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:12:46.200117 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 20:12:46.200158 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":7,"spinlock_wait_cycles":384,"thread_start_us":104,"threads_started":1}
I20260807 20:12:46.200521 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.196125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47932 (local address 127.23.200.195:40627)
0807 20:12:46.196288 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:46.196294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:46.196353 (+ 59us) server_negotiation.cc:407] Connection header received
0807 20:12:46.196430 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:46.196431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:46.196445 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:46.196503 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:46.196821 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.198041 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.199273 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.199760 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.199825 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.199884 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:46.200398 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 20:12:46.200446 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260807 20:12:46.201031 25448 tablet_service.cc:1467] Processing CreateTablet for tablet 03fbe503e6d543c09865d1999487c183 (DEFAULT_TABLE table=test-table [id=524ff171ec764c41a6c2a2e71b9c20f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:46.201187 25856 tablet_service.cc:1467] Processing CreateTablet for tablet 03fbe503e6d543c09865d1999487c183 (DEFAULT_TABLE table=test-table [id=524ff171ec764c41a6c2a2e71b9c20f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:46.201459 25448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03fbe503e6d543c09865d1999487c183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:46.201484 25856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03fbe503e6d543c09865d1999487c183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:46.201627 25970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.196070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36878 (local address 127.23.200.194:44385)
0807 20:12:46.196784 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:46.196790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:46.196800 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:46.196839 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:46.196841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:46.196857 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:46.197062 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:46.197155 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.197806 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.199242 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.199927 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.201428 (+ 1501us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.201478 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:46.201502 (+ 24us) server_negotiation.cc:299] Negotiation successful
0807 20:12:46.201548 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":81,"threads_started":1}
I20260807 20:12:46.202183 25653 tablet_service.cc:1467] Processing CreateTablet for tablet 03fbe503e6d543c09865d1999487c183 (DEFAULT_TABLE table=test-table [id=524ff171ec764c41a6c2a2e71b9c20f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:46.202481 25653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03fbe503e6d543c09865d1999487c183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:46.204874 25978 tablet_bootstrap.cc:492] T 03fbe503e6d543c09865d1999487c183 P 6f78f3e7448c4e99afdce3d0c7df250c: Bootstrap starting.
I20260807 20:12:46.205256 25977 tablet_bootstrap.cc:492] T 03fbe503e6d543c09865d1999487c183 P 1ba3fbf634724c89a294fcbc40d5f723: Bootstrap starting.
I20260807 20:12:46.205324 25981 tablet_bootstrap.cc:492] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270: Bootstrap starting.
I20260807 20:12:46.205914 25981 tablet_bootstrap.cc:654] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:46.205915 25977 tablet_bootstrap.cc:654] T 03fbe503e6d543c09865d1999487c183 P 1ba3fbf634724c89a294fcbc40d5f723: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:46.206508 25981 log.cc:824] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:46.206508 25977 log.cc:824] T 03fbe503e6d543c09865d1999487c183 P 1ba3fbf634724c89a294fcbc40d5f723: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:46.207207 25978 tablet_bootstrap.cc:654] T 03fbe503e6d543c09865d1999487c183 P 6f78f3e7448c4e99afdce3d0c7df250c: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:46.207304 25977 tablet_bootstrap.cc:492] T 03fbe503e6d543c09865d1999487c183 P 1ba3fbf634724c89a294fcbc40d5f723: No bootstrap required, opened a new log
I20260807 20:12:46.207384 25977 ts_tablet_manager.cc:1397] T 03fbe503e6d543c09865d1999487c183 P 1ba3fbf634724c89a294fcbc40d5f723: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:12:46.207444 25981 tablet_bootstrap.cc:492] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270: No bootstrap required, opened a new log
I20260807 20:12:46.207512 25981 ts_tablet_manager.cc:1397] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:12:46.207790 25978 log.cc:824] T 03fbe503e6d543c09865d1999487c183 P 6f78f3e7448c4e99afdce3d0c7df250c: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:46.208575 25978 tablet_bootstrap.cc:492] T 03fbe503e6d543c09865d1999487c183 P 6f78f3e7448c4e99afdce3d0c7df250c: No bootstrap required, opened a new log
I20260807 20:12:46.208642 25978 ts_tablet_manager.cc:1397] T 03fbe503e6d543c09865d1999487c183 P 6f78f3e7448c4e99afdce3d0c7df250c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 20:12:46.209367 25981 raft_consensus.cc:348] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f78f3e7448c4e99afdce3d0c7df250c" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40627 } } peers { permanent_uuid: "1ba3fbf634724c89a294fcbc40d5f723" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 35229 } } peers { permanent_uuid: "80fed330578d45a2af04af0e5266a270" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44385 } }
I20260807 20:12:46.209357 25977 raft_consensus.cc:348] T 03fbe503e6d543c09865d1999487c183 P 1ba3fbf634724c89a294fcbc40d5f723 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f78f3e7448c4e99afdce3d0c7df250c" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40627 } } peers { permanent_uuid: "1ba3fbf634724c89a294fcbc40d5f723" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 35229 } } peers { permanent_uuid: "80fed330578d45a2af04af0e5266a270" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44385 } }
I20260807 20:12:46.209524 25981 raft_consensus.cc:374] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:46.209532 25977 raft_consensus.cc:374] T 03fbe503e6d543c09865d1999487c183 P 1ba3fbf634724c89a294fcbc40d5f723 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:46.209560 25981 raft_consensus.cc:729] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80fed330578d45a2af04af0e5266a270, State: Initialized, Role: FOLLOWER
I20260807 20:12:46.209568 25977 raft_consensus.cc:729] T 03fbe503e6d543c09865d1999487c183 P 1ba3fbf634724c89a294fcbc40d5f723 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ba3fbf634724c89a294fcbc40d5f723, State: Initialized, Role: FOLLOWER
I20260807 20:12:46.209681 25981 consensus_queue.cc:260] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f78f3e7448c4e99afdce3d0c7df250c" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40627 } } peers { permanent_uuid: "1ba3fbf634724c89a294fcbc40d5f723" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 35229 } } peers { permanent_uuid: "80fed330578d45a2af04af0e5266a270" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44385 } }
I20260807 20:12:46.209764 25977 consensus_queue.cc:260] T 03fbe503e6d543c09865d1999487c183 P 1ba3fbf634724c89a294fcbc40d5f723 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f78f3e7448c4e99afdce3d0c7df250c" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40627 } } peers { permanent_uuid: "1ba3fbf634724c89a294fcbc40d5f723" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 35229 } } peers { permanent_uuid: "80fed330578d45a2af04af0e5266a270" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44385 } }
I20260807 20:12:46.209971 25981 ts_tablet_manager.cc:1428] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 20:12:46.210004 25747 heartbeater.cc:503] Master 127.23.200.254:34887 was elected leader, sending a full tablet report...
I20260807 20:12:46.210355 25977 ts_tablet_manager.cc:1428] T 03fbe503e6d543c09865d1999487c183 P 1ba3fbf634724c89a294fcbc40d5f723: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:12:46.210515 25542 heartbeater.cc:503] Master 127.23.200.254:34887 was elected leader, sending a full tablet report...
I20260807 20:12:46.210664 25978 raft_consensus.cc:348] T 03fbe503e6d543c09865d1999487c183 P 6f78f3e7448c4e99afdce3d0c7df250c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f78f3e7448c4e99afdce3d0c7df250c" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40627 } } peers { permanent_uuid: "1ba3fbf634724c89a294fcbc40d5f723" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 35229 } } peers { permanent_uuid: "80fed330578d45a2af04af0e5266a270" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44385 } }
I20260807 20:12:46.210824 25978 raft_consensus.cc:374] T 03fbe503e6d543c09865d1999487c183 P 6f78f3e7448c4e99afdce3d0c7df250c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:46.210877 25978 raft_consensus.cc:729] T 03fbe503e6d543c09865d1999487c183 P 6f78f3e7448c4e99afdce3d0c7df250c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f78f3e7448c4e99afdce3d0c7df250c, State: Initialized, Role: FOLLOWER
I20260807 20:12:46.211015 25978 consensus_queue.cc:260] T 03fbe503e6d543c09865d1999487c183 P 6f78f3e7448c4e99afdce3d0c7df250c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f78f3e7448c4e99afdce3d0c7df250c" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40627 } } peers { permanent_uuid: "1ba3fbf634724c89a294fcbc40d5f723" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 35229 } } peers { permanent_uuid: "80fed330578d45a2af04af0e5266a270" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44385 } }
I20260807 20:12:46.211277 25978 ts_tablet_manager.cc:1428] T 03fbe503e6d543c09865d1999487c183 P 6f78f3e7448c4e99afdce3d0c7df250c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:12:46.211297 25949 heartbeater.cc:503] Master 127.23.200.254:34887 was elected leader, sending a full tablet report...
W20260807 20:12:46.254781 25748 tablet.cc:2369] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:12:46.328591 25544 tablet.cc:2369] T 03fbe503e6d543c09865d1999487c183 P 1ba3fbf634724c89a294fcbc40d5f723: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:12:46.414793 25950 tablet.cc:2369] T 03fbe503e6d543c09865d1999487c183 P 6f78f3e7448c4e99afdce3d0c7df250c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:12:46.527829 25988 raft_consensus.cc:482] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:12:46.527967 25988 raft_consensus.cc:504] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f78f3e7448c4e99afdce3d0c7df250c" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40627 } } peers { permanent_uuid: "1ba3fbf634724c89a294fcbc40d5f723" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 35229 } } peers { permanent_uuid: "80fed330578d45a2af04af0e5266a270" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44385 } }
I20260807 20:12:46.528286 25988 leader_election.cc:288] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f78f3e7448c4e99afdce3d0c7df250c (127.23.200.195:40627), 1ba3fbf634724c89a294fcbc40d5f723 (127.23.200.193:35229)
I20260807 20:12:46.531126 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.528582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:48587 (local address 127.23.200.195:40627)
0807 20:12:46.528641 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:46.528644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:46.528656 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:12:46.528723 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:46.528724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:46.528734 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:46.528804 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:46.528995 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.529515 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.530448 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.530815 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.530876 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.530885 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:46.531025 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 20:12:46.531062 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:12:46.531131 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.528474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:40627 (local address 127.23.200.194:48587)
0807 20:12:46.528592 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:46.528604 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:46.528661 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:46.528830 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:46.528831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:46.528835 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:46.528924 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:46.528926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.529588 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.529589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:46.530352 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:46.530354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.530891 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.530892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:46.530962 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.530965 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:12:46.531010 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 20:12:46.531050 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":1,"thread_start_us":61,"threads_started":1}
I20260807 20:12:46.531198 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.528714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:37149 (local address 127.23.200.193:35229)
0807 20:12:46.528758 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:46.528759 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:46.528834 (+ 75us) server_negotiation.cc:407] Connection header received
0807 20:12:46.528883 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:46.528884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:46.528894 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:46.528934 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:46.529077 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.529561 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.530540 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.530890 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.530955 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.530963 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:46.531091 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 20:12:46.531133 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:12:46.531575 25881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03fbe503e6d543c09865d1999487c183" candidate_uuid: "80fed330578d45a2af04af0e5266a270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f78f3e7448c4e99afdce3d0c7df250c" is_pre_election: true
I20260807 20:12:46.531776 25881 raft_consensus.cc:2454] T 03fbe503e6d543c09865d1999487c183 P 6f78f3e7448c4e99afdce3d0c7df250c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80fed330578d45a2af04af0e5266a270 in term 0.
I20260807 20:12:46.531379 25997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.528647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:35229 (local address 127.23.200.194:37149)
0807 20:12:46.528771 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:46.528779 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:46.528831 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:46.528941 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:46.528942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:46.528944 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:46.529020 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:46.529022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.529633 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.529634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:46.530455 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:46.530458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.530956 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.530957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:46.531034 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.531037 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:12:46.531081 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 20:12:46.531118 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260807 20:12:46.532055 25579 leader_election.cc:302] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f78f3e7448c4e99afdce3d0c7df250c, 80fed330578d45a2af04af0e5266a270; no voters:
I20260807 20:12:46.532174 25988 raft_consensus.cc:2781] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:12:46.532253 25988 raft_consensus.cc:482] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:12:46.532275 25988 raft_consensus.cc:3037] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:46.532382 25478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03fbe503e6d543c09865d1999487c183" candidate_uuid: "80fed330578d45a2af04af0e5266a270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ba3fbf634724c89a294fcbc40d5f723" is_pre_election: true
I20260807 20:12:46.532513 25478 raft_consensus.cc:2454] T 03fbe503e6d543c09865d1999487c183 P 1ba3fbf634724c89a294fcbc40d5f723 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80fed330578d45a2af04af0e5266a270 in term 0.
I20260807 20:12:46.533038 25988 raft_consensus.cc:504] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f78f3e7448c4e99afdce3d0c7df250c" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40627 } } peers { permanent_uuid: "1ba3fbf634724c89a294fcbc40d5f723" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 35229 } } peers { permanent_uuid: "80fed330578d45a2af04af0e5266a270" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44385 } }
I20260807 20:12:46.533167 25988 leader_election.cc:288] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [CANDIDATE]: Term 1 election: Requested vote from peers 6f78f3e7448c4e99afdce3d0c7df250c (127.23.200.195:40627), 1ba3fbf634724c89a294fcbc40d5f723 (127.23.200.193:35229)
I20260807 20:12:46.533332 25881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03fbe503e6d543c09865d1999487c183" candidate_uuid: "80fed330578d45a2af04af0e5266a270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f78f3e7448c4e99afdce3d0c7df250c"
I20260807 20:12:46.533385 25478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03fbe503e6d543c09865d1999487c183" candidate_uuid: "80fed330578d45a2af04af0e5266a270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ba3fbf634724c89a294fcbc40d5f723"
I20260807 20:12:46.533416 25881 raft_consensus.cc:3037] T 03fbe503e6d543c09865d1999487c183 P 6f78f3e7448c4e99afdce3d0c7df250c [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:46.533468 25478 raft_consensus.cc:3037] T 03fbe503e6d543c09865d1999487c183 P 1ba3fbf634724c89a294fcbc40d5f723 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:46.534149 25881 raft_consensus.cc:2454] T 03fbe503e6d543c09865d1999487c183 P 6f78f3e7448c4e99afdce3d0c7df250c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80fed330578d45a2af04af0e5266a270 in term 1.
I20260807 20:12:46.534152 25478 raft_consensus.cc:2454] T 03fbe503e6d543c09865d1999487c183 P 1ba3fbf634724c89a294fcbc40d5f723 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80fed330578d45a2af04af0e5266a270 in term 1.
I20260807 20:12:46.534325 25579 leader_election.cc:302] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f78f3e7448c4e99afdce3d0c7df250c, 80fed330578d45a2af04af0e5266a270; no voters:
I20260807 20:12:46.534468 25988 raft_consensus.cc:2781] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:46.534636 25988 raft_consensus.cc:686] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [term 1 LEADER]: Becoming Leader. State: Replica: 80fed330578d45a2af04af0e5266a270, State: Running, Role: LEADER
I20260807 20:12:46.534739 25988 consensus_queue.cc:237] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f78f3e7448c4e99afdce3d0c7df250c" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40627 } } peers { permanent_uuid: "1ba3fbf634724c89a294fcbc40d5f723" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 35229 } } peers { permanent_uuid: "80fed330578d45a2af04af0e5266a270" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44385 } }
I20260807 20:12:46.535557 25259 catalog_manager.cc:5495] T 03fbe503e6d543c09865d1999487c183 P 80fed330578d45a2af04af0e5266a270 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80fed330578d45a2af04af0e5266a270 (127.23.200.194). New cstate: current_term: 1 leader_uuid: "80fed330578d45a2af04af0e5266a270" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f78f3e7448c4e99afdce3d0c7df250c" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ba3fbf634724c89a294fcbc40d5f723" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 35229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80fed330578d45a2af04af0e5266a270" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44385 } health_report { overall_health: HEALTHY } } }
Aug 07 20:12:46 dist-test-slave-2j7r krb5kdc[25193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.193@KRBTEST.COM
I20260807 20:12:46.557370 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.550002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50770 (local address 127.23.200.193:35229)
0807 20:12:46.550059 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:46.550061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:46.550162 (+ 101us) server_negotiation.cc:407] Connection header received
0807 20:12:46.550232 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:46.550233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:46.550243 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:46.550292 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:46.550882 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.551370 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.552054 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.552242 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.552295 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.554663 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:46.554679 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:46.554680 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:46.554688 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:46.556194 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:46.556559 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:46.556564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:46.556564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:46.556630 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:46.556813 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:46.556813 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:46.556814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:46.556962 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:46.557131 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:46.557189 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 20:12:46.557274 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Aug 07 20:12:46 dist-test-slave-2j7r krb5kdc[25193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133566, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 20:12:46 dist-test-slave-2j7r krb5kdc[25193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133566, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.200.193@KRBTEST.COM
I20260807 20:12:46.580880 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.574074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50784 (local address 127.23.200.193:35229)
0807 20:12:46.574138 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:46.574140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:46.574228 (+ 88us) server_negotiation.cc:407] Connection header received
0807 20:12:46.574292 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:46.574293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:46.574303 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:46.574349 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:46.574947 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.575410 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.576099 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.576285 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.576338 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.578430 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:46.578431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:46.578432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:46.578438 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:46.579790 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:46.580114 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:46.580115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:46.580115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:46.580171 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:46.580400 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:46.580402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:46.580402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:46.580506 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:46.580588 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:46.580751 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 20:12:46.580814 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
Aug 07 20:12:46 dist-test-slave-2j7r krb5kdc[25193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 20:12:46 dist-test-slave-2j7r krb5kdc[25193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.193@KRBTEST.COM
I20260807 20:12:46.603175 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:46.596690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50790 (local address 127.23.200.193:35229)
0807 20:12:46.596752 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:46.596754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:46.596813 (+ 59us) server_negotiation.cc:407] Connection header received
0807 20:12:46.596880 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:46.596882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:46.596893 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:46.596942 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:46.597537 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.598000 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.598683 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:46.598864 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:46.598918 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:46.600984 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:46.600985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:46.600985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:46.600993 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:46.602202 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:46.602513 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:46.602515 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:46.602515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:46.602565 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:46.602723 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:46.602724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:46.602724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:46.602816 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:46.602900 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:46.603008 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 20:12:46.603082 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 20:12:46.603773 25414 tablet_service.cc:2908] Rejecting scan request for tablet 03fbe503e6d543c09865d1999487c183: Uninitialized: clean time has not yet been initialized
I20260807 20:12:46.604732 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 25350
I20260807 20:12:46.617730 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 25549
I20260807 20:12:46.632846 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 25756
I20260807 20:12:46.643462 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 25213
2026-08-07T20:12:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2443 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:46 dist-test-slave-2j7r krb5kdc[26021](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:46 dist-test-slave-2j7r krb5kdc[26021](info): set up 2 sockets
Aug 07 20:12:46 dist-test-slave-2j7r krb5kdc[26021](info): commencing operation
krb5kdc: starting...
W20260807 20:12:47.807298 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:12:47 dist-test-slave-2j7r krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:12:47.929299 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:36021
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:36727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:36021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:48.011709 26037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:48.011937 26037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:48.011966 26037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:48.013164 26037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:48.013206 26037 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 20:12:48.013222 26037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:48.013239 26037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:48.013254 26037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:48.013270 26037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:48.014798 26037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:36727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:36021
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:36021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:48.015046 26037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:48.015264 26037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:48.020069 26045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:48.020228 26037 server_base.cc:1034] running on GCE node
W20260807 20:12:48.020079 26048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:48.020094 26043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:48.020598 26037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:48.020884 26037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:48.022032 26037 hybrid_clock.cc:648] HybridClock initialized: now 1786133568022001 us; error 44 us; skew 500 ppm
Aug 07 20:12:48 dist-test-slave-2j7r krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133568, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:48.024569 26037 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:12:48.025019 26037 webserver.cc:460] Webserver started at http://127.23.200.254:38189/ using document root <none> and password file <none>
I20260807 20:12:48.025245 26037 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:48.025295 26037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:48.025421 26037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:48.026549 26037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "81acf40261e545e7b50ab33e40b9f4be"
format_stamp: "Formatted at 2026-08-07 20:12:48 on dist-test-slave-2j7r"
server_key: "f5b37b0260284b672de52f512f2097ee"
server_key_iv: "a4f6b00a2fc9f3975726055acfe7718b"
server_key_version: "encryptionkey@0"
I20260807 20:12:48.026926 26037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "81acf40261e545e7b50ab33e40b9f4be"
format_stamp: "Formatted at 2026-08-07 20:12:48 on dist-test-slave-2j7r"
server_key: "f5b37b0260284b672de52f512f2097ee"
server_key_iv: "a4f6b00a2fc9f3975726055acfe7718b"
server_key_version: "encryptionkey@0"
I20260807 20:12:48.028204 26037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:48.028877 26058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:48.029059 26037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:48.029170 26037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "81acf40261e545e7b50ab33e40b9f4be"
format_stamp: "Formatted at 2026-08-07 20:12:48 on dist-test-slave-2j7r"
server_key: "f5b37b0260284b672de52f512f2097ee"
server_key_iv: "a4f6b00a2fc9f3975726055acfe7718b"
server_key_version: "encryptionkey@0"
I20260807 20:12:48.029233 26037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:48.037254 26037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:48.038954 26037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:48.039081 26037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:48.043439 26037 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:36021
I20260807 20:12:48.043504 26151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:36021 every 8 connection(s)
I20260807 20:12:48.043787 26037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:48.044340 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 26037
I20260807 20:12:48.044437 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:12:48.044472 26152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:48.044601 24355 external_mini_cluster.cc:1428] Setting key df9951284a02614d07cf057b050abdc4
I20260807 20:12:48.047818 26152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be: Bootstrap starting.
I20260807 20:12:48.048516 26152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:48.048801 26152 log.cc:824] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:48.049547 26152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be: No bootstrap required, opened a new log
Aug 07 20:12:48 dist-test-slave-2j7r krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:48.051435 26152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81acf40261e545e7b50ab33e40b9f4be" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 36021 } }
I20260807 20:12:48.051568 26152 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:48.051601 26152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81acf40261e545e7b50ab33e40b9f4be, State: Initialized, Role: FOLLOWER
I20260807 20:12:48.051728 26152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81acf40261e545e7b50ab33e40b9f4be" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 36021 } }
I20260807 20:12:48.051811 26152 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:48.051851 26152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:48.051898 26152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:48.052515 26152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81acf40261e545e7b50ab33e40b9f4be" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 36021 } }
I20260807 20:12:48.052621 26152 leader_election.cc:302] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81acf40261e545e7b50ab33e40b9f4be; no voters:
I20260807 20:12:48.052814 26152 leader_election.cc:288] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:48.052865 26162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:48.053013 26162 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [term 1 LEADER]: Becoming Leader. State: Replica: 81acf40261e545e7b50ab33e40b9f4be, State: Running, Role: LEADER
I20260807 20:12:48.053103 26152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:48.053171 26162 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81acf40261e545e7b50ab33e40b9f4be" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 36021 } }
I20260807 20:12:48.053570 26162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81acf40261e545e7b50ab33e40b9f4be. Latest consensus state: current_term: 1 leader_uuid: "81acf40261e545e7b50ab33e40b9f4be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81acf40261e545e7b50ab33e40b9f4be" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 36021 } } }
I20260807 20:12:48.053684 26162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:48.053573 26166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81acf40261e545e7b50ab33e40b9f4be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81acf40261e545e7b50ab33e40b9f4be" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 36021 } } }
I20260807 20:12:48.053776 26166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:48.053947 26175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:48.054553 26175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:48.055704 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.045131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38504 (local address 127.23.200.254:36021)
0807 20:12:48.045509 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.045519 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.045553 (+ 34us) server_negotiation.cc:407] Connection header received
0807 20:12:48.046149 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.046158 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.046225 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.046342 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:48.047089 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.048054 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.049109 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.049522 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.049619 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.051770 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:48.051774 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:48.051775 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:48.051788 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:48.053723 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.054148 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.054151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.054151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.054256 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.054490 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.054490 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.054491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.054792 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:48.055282 (+ 490us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.055358 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.055506 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":113,"threads_started":1}
I20260807 20:12:48.056089 26175 catalog_manager.cc:1347] Generated new cluster ID: dbff6b15308d4e55868e9f6f865f3749
I20260807 20:12:48.056139 26175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:48.077148 26175 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:48.077901 26175 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:48.099459 26175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be: Generated new TSK 0
I20260807 20:12:48.099677 26175 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:12:48.144678 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:36021
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:36727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:48.243444 26193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:48.243651 26193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:48.243675 26193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:48.244810 26193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:48.244853 26193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:48.244871 26193 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:12:48.244907 26193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:12:48.246379 26193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:36727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.200.254:36021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:48.246615 26193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:48.246835 26193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:48.251724 26199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:48.251745 26202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:48.251745 26200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:48.251821 26193 server_base.cc:1034] running on GCE node
I20260807 20:12:48.252127 26193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:48.252341 26193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:48.253539 26193 hybrid_clock.cc:648] HybridClock initialized: now 1786133568253525 us; error 30 us; skew 500 ppm
Aug 07 20:12:48 dist-test-slave-2j7r krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133568, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:48.256155 26193 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:12:48.256630 26193 webserver.cc:460] Webserver started at http://127.23.200.193:41417/ using document root <none> and password file <none>
I20260807 20:12:48.256855 26193 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:48.256903 26193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:48.257028 26193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:48.258083 26193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "e1545a2943db4e799aa04fc4860b99d8"
format_stamp: "Formatted at 2026-08-07 20:12:48 on dist-test-slave-2j7r"
server_key: "e7d453159361a70e77290cd430786b93"
server_key_iv: "0d1f671621d49c171eeb7b95b14ca8ee"
server_key_version: "encryptionkey@0"
I20260807 20:12:48.258438 26193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "e1545a2943db4e799aa04fc4860b99d8"
format_stamp: "Formatted at 2026-08-07 20:12:48 on dist-test-slave-2j7r"
server_key: "e7d453159361a70e77290cd430786b93"
server_key_iv: "0d1f671621d49c171eeb7b95b14ca8ee"
server_key_version: "encryptionkey@0"
I20260807 20:12:48.259644 26193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:12:48.260319 26214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:48.260478 26193 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:48.260563 26193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "e1545a2943db4e799aa04fc4860b99d8"
format_stamp: "Formatted at 2026-08-07 20:12:48 on dist-test-slave-2j7r"
server_key: "e7d453159361a70e77290cd430786b93"
server_key_iv: "0d1f671621d49c171eeb7b95b14ca8ee"
server_key_version: "encryptionkey@0"
I20260807 20:12:48.260623 26193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:48.267323 26193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:48.269027 26193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:48.269183 26193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:48.269673 26193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:48.269712 26193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:48.269747 26193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:48.269774 26193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 20:12:48 dist-test-slave-2j7r krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133568, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:48.276512 26193 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:42097
I20260807 20:12:48.276588 26392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:42097 every 8 connection(s)
I20260807 20:12:48.276933 26193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:12:48.279047 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.270161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:43229 (local address 127.23.200.254:36021)
0807 20:12:48.270226 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.270228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.270413 (+ 185us) server_negotiation.cc:407] Connection header received
0807 20:12:48.271051 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.271052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.271063 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.271103 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:48.271911 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.272701 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.274249 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.274507 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.274563 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.276579 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:48.276580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:48.276580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:48.276589 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:48.277937 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.278279 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.278279 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.278279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.278324 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.278513 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.278513 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.278513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.278606 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:48.278649 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.278886 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.278982 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 20:12:48.279243 26227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.270126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:36021 (local address 127.23.200.193:43229)
0807 20:12:48.270270 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:48.270318 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:48.270940 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:48.271173 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:48.271177 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:48.271523 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:48.271805 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.271812 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.272810 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.272811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.274150 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.274153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.274608 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.274610 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.274716 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.275129 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:48.275137 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:48.276479 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:48.278003 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:48.278009 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:48.278011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:48.278222 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:48.278367 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:48.278368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:48.278368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:48.278469 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:48.278647 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:48.278650 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:48.278780 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 20:12:48.278877 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 20:12:48.279035 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
I20260807 20:12:48.279665 26393 heartbeater.cc:347] Connected to a master server at 127.23.200.254:36021
I20260807 20:12:48.279747 26393 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:48.279913 26393 heartbeater.cc:511] Master 127.23.200.254:36021 requested a full tablet report, sending...
I20260807 20:12:48.280439 26083 ts_manager.cc:194] Registered new tserver with Master: e1545a2943db4e799aa04fc4860b99d8 (127.23.200.193:42097)
I20260807 20:12:48.281210 26083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:43229
I20260807 20:12:48.281446 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 26193
I20260807 20:12:48.281543 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:12:48.281719 24355 external_mini_cluster.cc:1428] Setting key cdfe793fb94b8d245d0326fe1a5241b9
I20260807 20:12:48.285936 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.280775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:45101 (local address 127.23.200.254:36021)
0807 20:12:48.280812 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.280814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.280964 (+ 150us) server_negotiation.cc:407] Connection header received
0807 20:12:48.281016 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.281017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.281027 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.281071 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:48.281392 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.282067 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.282784 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.282978 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.283028 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.283792 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:48.283793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:48.283793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:48.283800 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:48.284961 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.285283 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.285284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.285284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.285338 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.285496 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.285496 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.285497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.285595 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:48.285649 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.285781 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.285847 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:12:48.285971 26404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.280719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:36021 (local address 127.23.200.193:45101)
0807 20:12:48.280878 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:48.280886 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:48.280964 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:48.281073 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:48.281074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:48.281265 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:48.281323 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.281325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.282150 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.282151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.282715 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.282717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.283033 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.283034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.283109 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.283468 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:48.283471 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:48.283717 (+ 246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:48.285034 (+ 1317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:48.285034 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:48.285035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:48.285224 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:48.285390 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:48.285390 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:48.285391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:48.285446 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:48.285648 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:48.285649 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:48.285692 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 20:12:48.285750 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 20:12:48.285850 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:12:48.305684 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:36021
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:36727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:48.387959 26408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:48.388165 26408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:48.388204 26408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:48.389424 26408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:48.389465 26408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:48.389492 26408 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:12:48.389526 26408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:12:48.391026 26408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:36727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.200.254:36021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:48.391285 26408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:48.391512 26408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:48.396481 26423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:48.396549 26420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:48.396595 26408 server_base.cc:1034] running on GCE node
W20260807 20:12:48.396560 26421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:48.396931 26408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:48.397217 26408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:48.398358 26408 hybrid_clock.cc:648] HybridClock initialized: now 1786133568398344 us; error 31 us; skew 500 ppm
Aug 07 20:12:48 dist-test-slave-2j7r krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133568, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:48.400691 26408 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:12:48.401155 26408 webserver.cc:460] Webserver started at http://127.23.200.194:44517/ using document root <none> and password file <none>
I20260807 20:12:48.401401 26408 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:48.401451 26408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:48.401573 26408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:48.402604 26408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "345d68f75ab3496b991b822920e9686c"
format_stamp: "Formatted at 2026-08-07 20:12:48 on dist-test-slave-2j7r"
server_key: "2e3a51a97397175366a38efaf4f8a2b6"
server_key_iv: "5415554063a72d1d9ac89b8a364df036"
server_key_version: "encryptionkey@0"
I20260807 20:12:48.402944 26408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "345d68f75ab3496b991b822920e9686c"
format_stamp: "Formatted at 2026-08-07 20:12:48 on dist-test-slave-2j7r"
server_key: "2e3a51a97397175366a38efaf4f8a2b6"
server_key_iv: "5415554063a72d1d9ac89b8a364df036"
server_key_version: "encryptionkey@0"
I20260807 20:12:48.404197 26408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:12:48.404899 26433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:48.405061 26408 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:48.405170 26408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "345d68f75ab3496b991b822920e9686c"
format_stamp: "Formatted at 2026-08-07 20:12:48 on dist-test-slave-2j7r"
server_key: "2e3a51a97397175366a38efaf4f8a2b6"
server_key_iv: "5415554063a72d1d9ac89b8a364df036"
server_key_version: "encryptionkey@0"
I20260807 20:12:48.405231 26408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:48.412566 26408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:48.414278 26408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:48.414414 26408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:48.414882 26408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:48.414932 26408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:48.414960 26408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:48.414978 26408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:48.421284 26408 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:45443
I20260807 20:12:48.421363 26609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:45443 every 8 connection(s)
Aug 07 20:12:48 dist-test-slave-2j7r krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133568, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:48.421672 26408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:12:48.424340 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.415365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:39785 (local address 127.23.200.254:36021)
0807 20:12:48.415410 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.415412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.415662 (+ 250us) server_negotiation.cc:407] Connection header received
0807 20:12:48.416192 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.416194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.416210 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.416263 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:48.417128 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.417892 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.419350 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.419662 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.419744 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.422144 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:48.422146 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:48.422146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:48.422155 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:48.423256 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.423640 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.423641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.423641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.423690 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.423866 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.423867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.423867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.423963 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:48.424009 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.424212 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.424274 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:12:48.424577 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.415371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:36021 (local address 127.23.200.194:39785)
0807 20:12:48.415545 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:48.415581 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:48.416083 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:48.416349 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:48.416354 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:48.416706 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:48.417006 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.417013 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.417991 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.417992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.419259 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.419262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.419742 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.419743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.419844 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.420386 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:48.420398 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:48.422053 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:48.423327 (+ 1274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:48.423334 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:48.423335 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:48.423580 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:48.423735 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:48.423735 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:48.423736 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:48.423826 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:48.424005 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:48.424008 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:48.424119 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 20:12:48.424195 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 20:12:48.424315 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
I20260807 20:12:48.425056 26612 heartbeater.cc:347] Connected to a master server at 127.23.200.254:36021
I20260807 20:12:48.425171 26612 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:48.425365 26612 heartbeater.cc:511] Master 127.23.200.254:36021 requested a full tablet report, sending...
I20260807 20:12:48.425783 26084 ts_manager.cc:194] Registered new tserver with Master: 345d68f75ab3496b991b822920e9686c (127.23.200.194:45443)
I20260807 20:12:48.426213 26084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:39785
I20260807 20:12:48.430439 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 26408
I20260807 20:12:48.430514 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:12:48.430680 24355 external_mini_cluster.cc:1428] Setting key 04107b8359bd3d794c89a4d0ded2889c
I20260807 20:12:48.431296 26623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.426060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:36021 (local address 127.23.200.194:35659)
0807 20:12:48.426175 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:48.426185 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:48.426237 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:48.426355 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:48.426356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:48.426570 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:48.426623 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.426625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.427269 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.427270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.427776 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.427777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.428105 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.428105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.428184 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.428571 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:48.428576 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:48.428835 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:48.430210 (+ 1375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:48.430211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:48.430211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:48.430385 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:48.430593 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:48.430594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:48.430594 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:48.430652 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:48.430975 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:48.430976 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:48.431029 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 20:12:48.431100 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 20:12:48.431189 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":64,"threads_started":1}
I20260807 20:12:48.431311 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.426124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:35659 (local address 127.23.200.254:36021)
0807 20:12:48.426161 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.426163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.426229 (+ 66us) server_negotiation.cc:407] Connection header received
0807 20:12:48.426291 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.426291 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.426301 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.426349 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:48.426699 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.427179 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.427846 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.428048 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.428101 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.428921 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:48.428922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:48.428923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:48.428930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:48.430136 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.430480 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.430481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.430482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.430543 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.430705 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.430705 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.430706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.430905 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:48.430970 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.431145 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.431230 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:12:48.455494 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:36021
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:36727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:48.539422 26626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:48.539630 26626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:48.539664 26626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:48.540845 26626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:48.540887 26626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:48.540915 26626 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:12:48.540939 26626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:12:48.542431 26626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:36727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.200.254:36021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:48.542670 26626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:48.542884 26626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:48.547978 26638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:48.548067 26626 server_base.cc:1034] running on GCE node
W20260807 20:12:48.548012 26636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:48.548029 26633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:48.548457 26626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:48.548681 26626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:48.549819 26626 hybrid_clock.cc:648] HybridClock initialized: now 1786133568549805 us; error 27 us; skew 500 ppm
Aug 07 20:12:48 dist-test-slave-2j7r krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133568, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:48.552258 26626 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:12:48.552702 26626 webserver.cc:460] Webserver started at http://127.23.200.195:35883/ using document root <none> and password file <none>
I20260807 20:12:48.552935 26626 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:48.552979 26626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:48.553068 26626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:48.554226 26626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "3fdf7e0f624f4f3496458a93316ebda9"
format_stamp: "Formatted at 2026-08-07 20:12:48 on dist-test-slave-2j7r"
server_key: "8ba718cd680f1d6db6b71b4e5919fba9"
server_key_iv: "293a83c2392663024ee95655fdb9a0a4"
server_key_version: "encryptionkey@0"
I20260807 20:12:48.554538 26626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "3fdf7e0f624f4f3496458a93316ebda9"
format_stamp: "Formatted at 2026-08-07 20:12:48 on dist-test-slave-2j7r"
server_key: "8ba718cd680f1d6db6b71b4e5919fba9"
server_key_iv: "293a83c2392663024ee95655fdb9a0a4"
server_key_version: "encryptionkey@0"
I20260807 20:12:48.555932 26626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:48.556676 26650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:48.556833 26626 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:48.556921 26626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "3fdf7e0f624f4f3496458a93316ebda9"
format_stamp: "Formatted at 2026-08-07 20:12:48 on dist-test-slave-2j7r"
server_key: "8ba718cd680f1d6db6b71b4e5919fba9"
server_key_iv: "293a83c2392663024ee95655fdb9a0a4"
server_key_version: "encryptionkey@0"
I20260807 20:12:48.556980 26626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:48.573454 26626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:48.575423 26626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:48.575554 26626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:48.576056 26626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:48.576094 26626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:48.576122 26626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:48.576148 26626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:48.583010 26626 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:36587
I20260807 20:12:48.583081 26825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:36587 every 8 connection(s)
Aug 07 20:12:48 dist-test-slave-2j7r krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133568, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:48.583415 26626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:12:48.586277 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.576494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:47919 (local address 127.23.200.254:36021)
0807 20:12:48.576542 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.576545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.576755 (+ 210us) server_negotiation.cc:407] Connection header received
0807 20:12:48.577269 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.577271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.577286 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.577363 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:48.577978 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.578731 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.580298 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.580601 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.580670 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.583693 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:48.583695 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:48.583695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:48.583703 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:48.584981 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.585418 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.585419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.585420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.585486 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.585712 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.585712 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.585713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.585827 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:48.585874 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.586144 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.586211 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 20:12:48.586404 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.576495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:36021 (local address 127.23.200.195:47919)
0807 20:12:48.576638 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:48.576673 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:48.577176 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:48.577445 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:48.577449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:48.577677 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:48.577889 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.577894 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.578832 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.578833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.580195 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.580199 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.580699 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.580700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.580856 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.581894 (+ 1038us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:48.581906 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:48.583599 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:48.585071 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:48.585078 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:48.585080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:48.585331 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:48.585555 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:48.585556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:48.585556 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:48.585653 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:48.585873 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:48.585877 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:48.586020 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 20:12:48.586101 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 20:12:48.586224 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":82,"threads_started":1}
I20260807 20:12:48.586879 26827 heartbeater.cc:347] Connected to a master server at 127.23.200.254:36021
I20260807 20:12:48.586971 26827 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:48.587144 26827 heartbeater.cc:511] Master 127.23.200.254:36021 requested a full tablet report, sending...
I20260807 20:12:48.587505 26083 ts_manager.cc:194] Registered new tserver with Master: 3fdf7e0f624f4f3496458a93316ebda9 (127.23.200.195:36587)
I20260807 20:12:48.587977 26083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:47919
I20260807 20:12:48.591110 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 26626
I20260807 20:12:48.591203 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:12:48.591377 24355 external_mini_cluster.cc:1428] Setting key a18d32e7422537479c9d31647333d183
I20260807 20:12:48.592180 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:12:48.593014 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.587854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:36021 (local address 127.23.200.195:39307)
0807 20:12:48.587998 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:48.588007 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:48.588064 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:48.588189 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:48.588190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:48.588355 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:48.588408 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.588410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.589034 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.589035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.589578 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.589580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.589909 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.589910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.589983 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.590294 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:48.590298 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:48.590550 (+ 252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:48.592086 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:48.592088 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:48.592088 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:48.592299 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:48.592487 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:48.592487 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:48.592488 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:48.592539 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:48.592766 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:48.592766 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:48.592800 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 20:12:48.592842 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 20:12:48.592919 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260807 20:12:48.593022 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.587944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:39307 (local address 127.23.200.254:36021)
0807 20:12:48.587995 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.587997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.588051 (+ 54us) server_negotiation.cc:407] Connection header received
0807 20:12:48.588109 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.588110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.588120 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.588172 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:48.588484 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.588945 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.589649 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.589845 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.589898 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.590627 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:48.590628 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:48.590628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:48.590635 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:48.592011 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.592364 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.592365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.592365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.592430 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.592588 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.592588 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.592589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.592713 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:48.592777 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.592871 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.592945 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Aug 07 20:12:48 dist-test-slave-2j7r krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133568, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 20:12:48 dist-test-slave-2j7r krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133568, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:48.614538 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.608503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38510 (local address 127.23.200.254:36021)
0807 20:12:48.608557 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.608558 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.608567 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:12:48.608591 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.608591 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.608600 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.608646 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:48.609234 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.609711 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.610394 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.610568 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.610613 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.612595 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:48.612596 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:48.612596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:48.612603 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:48.613704 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.613993 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.613994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.613994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.614040 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.614176 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.614177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.614177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.614273 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:48.614341 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.614417 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.614474 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:12:48.615986 26083 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:12:48.617642 26083 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:12:48.623847 26852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.619780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:45443 (local address 127.0.0.1:55052)
0807 20:12:48.620343 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:48.620350 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:48.620388 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:48.620649 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:48.620649 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:48.620651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:48.620702 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.620704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.621616 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.621617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.622858 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.622861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.623557 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.623558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.623636 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.623649 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 20:12:48.623715 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 20:12:48.623768 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":3,"thread_start_us":79,"threads_started":1}
I20260807 20:12:48.623943 26856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.619989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:42097 (local address 127.0.0.1:33098)
0807 20:12:48.620454 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:48.620462 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:48.620502 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:48.620733 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:48.620734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:48.620736 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:48.620771 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.620772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.622003 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.622005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.622844 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.622846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.623716 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.623717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.623769 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.623773 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:12:48.623820 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 20:12:48.623851 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":73,"threads_started":1}
I20260807 20:12:48.624090 26861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.619780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:36587 (local address 127.0.0.1:32832)
0807 20:12:48.620127 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:48.620173 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:48.620250 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:48.620451 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:48.620455 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:48.620457 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:48.620526 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.620528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.621872 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.621873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.622940 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.622942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.623834 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.623835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.623906 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.623910 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:12:48.623965 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 20:12:48.624020 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":26,"thread_start_us":54,"threads_started":1}
I20260807 20:12:48.624185 26860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.619884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55052 (local address 127.23.200.194:45443)
0807 20:12:48.620104 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.620108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.620419 (+ 311us) server_negotiation.cc:407] Connection header received
0807 20:12:48.620472 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.620475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.620491 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.620579 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:48.620780 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.621541 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.622969 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.623478 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.623546 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.623607 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.624055 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.624097 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":105,"threads_started":1}
I20260807 20:12:48.624332 26859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.619891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32832 (local address 127.23.200.195:36587)
0807 20:12:48.620191 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.620195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.620226 (+ 31us) server_negotiation.cc:407] Connection header received
0807 20:12:48.620337 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.620340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.620355 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.620432 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:48.620611 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.621790 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.623015 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.623719 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.623812 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.623863 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.624191 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.624234 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":83,"threads_started":1}
I20260807 20:12:48.624576 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.620104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33098 (local address 127.23.200.193:42097)
0807 20:12:48.620268 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.620272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.620552 (+ 280us) server_negotiation.cc:407] Connection header received
0807 20:12:48.620595 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.620597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.620611 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.620673 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:48.620835 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.621936 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.622919 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.623626 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.623699 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.623757 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.624439 (+ 682us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.624483 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
I20260807 20:12:48.624873 26508 tablet_service.cc:1467] Processing CreateTablet for tablet 8f8cc735be69460586cf64c4ea8cc222 (DEFAULT_TABLE table=test-table [id=801d0b878471496791353e9bd66575c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:48.624936 26729 tablet_service.cc:1467] Processing CreateTablet for tablet 8f8cc735be69460586cf64c4ea8cc222 (DEFAULT_TABLE table=test-table [id=801d0b878471496791353e9bd66575c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:48.625286 26508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f8cc735be69460586cf64c4ea8cc222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:48.625286 26729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f8cc735be69460586cf64c4ea8cc222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:48.625630 26285 tablet_service.cc:1467] Processing CreateTablet for tablet 8f8cc735be69460586cf64c4ea8cc222 (DEFAULT_TABLE table=test-table [id=801d0b878471496791353e9bd66575c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:48.625979 26285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f8cc735be69460586cf64c4ea8cc222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:48.628629 26864 tablet_bootstrap.cc:492] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9: Bootstrap starting.
I20260807 20:12:48.629257 26865 tablet_bootstrap.cc:492] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8: Bootstrap starting.
I20260807 20:12:48.629487 26864 tablet_bootstrap.cc:654] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:48.629724 26864 log.cc:824] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:48.629846 26865 tablet_bootstrap.cc:654] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:48.630126 26865 log.cc:824] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:48.630355 26864 tablet_bootstrap.cc:492] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9: No bootstrap required, opened a new log
I20260807 20:12:48.630357 26866 tablet_bootstrap.cc:492] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c: Bootstrap starting.
I20260807 20:12:48.630417 26864 ts_tablet_manager.cc:1397] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 20:12:48.630841 26865 tablet_bootstrap.cc:492] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8: No bootstrap required, opened a new log
I20260807 20:12:48.630872 26866 tablet_bootstrap.cc:654] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:48.630911 26865 ts_tablet_manager.cc:1397] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:12:48.631148 26866 log.cc:824] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:48.631793 26866 tablet_bootstrap.cc:492] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c: No bootstrap required, opened a new log
I20260807 20:12:48.631848 26866 ts_tablet_manager.cc:1397] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 20:12:48.632330 26864 raft_consensus.cc:348] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } }
I20260807 20:12:48.632495 26864 raft_consensus.cc:374] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:48.632531 26864 raft_consensus.cc:729] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fdf7e0f624f4f3496458a93316ebda9, State: Initialized, Role: FOLLOWER
I20260807 20:12:48.632668 26864 consensus_queue.cc:260] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } }
I20260807 20:12:48.632962 26827 heartbeater.cc:503] Master 127.23.200.254:36021 was elected leader, sending a full tablet report...
I20260807 20:12:48.632862 26865 raft_consensus.cc:348] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } }
I20260807 20:12:48.633008 26865 raft_consensus.cc:374] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:48.633049 26865 raft_consensus.cc:729] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1545a2943db4e799aa04fc4860b99d8, State: Initialized, Role: FOLLOWER
I20260807 20:12:48.633181 26865 consensus_queue.cc:260] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } }
I20260807 20:12:48.633249 26864 ts_tablet_manager.cc:1428] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 20:12:48.633441 26865 ts_tablet_manager.cc:1428] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 20:12:48.633671 26393 heartbeater.cc:503] Master 127.23.200.254:36021 was elected leader, sending a full tablet report...
I20260807 20:12:48.634424 26866 raft_consensus.cc:348] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } }
I20260807 20:12:48.634539 26866 raft_consensus.cc:374] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:48.634558 26866 raft_consensus.cc:729] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345d68f75ab3496b991b822920e9686c, State: Initialized, Role: FOLLOWER
I20260807 20:12:48.634639 26866 consensus_queue.cc:260] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } }
I20260807 20:12:48.634845 26866 ts_tablet_manager.cc:1428] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260807 20:12:48.634902 26612 heartbeater.cc:503] Master 127.23.200.254:36021 was elected leader, sending a full tablet report...
W20260807 20:12:48.672250 26613 tablet.cc:2369] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:12:48.699167 26872 raft_consensus.cc:482] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:12:48.699299 26872 raft_consensus.cc:504] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } }
I20260807 20:12:48.699622 26872 leader_election.cc:288] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 345d68f75ab3496b991b822920e9686c (127.23.200.194:45443), e1545a2943db4e799aa04fc4860b99d8 (127.23.200.193:42097)
I20260807 20:12:48.702559 26860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.699866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:38657 (local address 127.23.200.194:45443)
0807 20:12:48.699918 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.699920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.699977 (+ 57us) server_negotiation.cc:407] Connection header received
0807 20:12:48.700067 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.700069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.700079 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.700132 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:48.700313 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.700829 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.701832 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.702209 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.702270 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.702280 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.702462 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.702501 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:12:48.702564 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.699978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:53993 (local address 127.23.200.193:42097)
0807 20:12:48.700021 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.700023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.700144 (+ 121us) server_negotiation.cc:407] Connection header received
0807 20:12:48.700231 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.700232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.700241 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.700300 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:48.700475 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.700955 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.701843 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.702216 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.702280 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.702288 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.702440 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.702478 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:12:48.702564 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.699806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:45443 (local address 127.23.200.195:38657)
0807 20:12:48.699884 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:48.699906 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:48.699979 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:48.700157 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:48.700159 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:48.700161 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:48.700235 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.700237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.700900 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.700901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.701741 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.701744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.702286 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.702287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.702373 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.702376 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:12:48.702448 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 20:12:48.702485 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 20:12:48.702565 26875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.699806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:42097 (local address 127.23.200.195:53993)
0807 20:12:48.700077 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:48.700086 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:48.700178 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:48.700310 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:48.700311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:48.700312 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:48.700414 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.700416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.701029 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.701030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.701758 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.701760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.702299 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.702300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.702368 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.702372 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:12:48.702421 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 20:12:48.702460 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":87,"thread_start_us":115,"threads_started":1}
I20260807 20:12:48.702945 26539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f8cc735be69460586cf64c4ea8cc222" candidate_uuid: "3fdf7e0f624f4f3496458a93316ebda9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "345d68f75ab3496b991b822920e9686c" is_pre_election: true
I20260807 20:12:48.703097 26539 raft_consensus.cc:2454] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fdf7e0f624f4f3496458a93316ebda9 in term 0.
I20260807 20:12:48.703049 26318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f8cc735be69460586cf64c4ea8cc222" candidate_uuid: "3fdf7e0f624f4f3496458a93316ebda9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1545a2943db4e799aa04fc4860b99d8" is_pre_election: true
I20260807 20:12:48.703212 26318 raft_consensus.cc:2454] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fdf7e0f624f4f3496458a93316ebda9 in term 0.
I20260807 20:12:48.703351 26656 leader_election.cc:302] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 345d68f75ab3496b991b822920e9686c, 3fdf7e0f624f4f3496458a93316ebda9; no voters:
I20260807 20:12:48.703504 26872 raft_consensus.cc:2781] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:12:48.703559 26872 raft_consensus.cc:482] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:12:48.703583 26872 raft_consensus.cc:3037] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:48.704277 26872 raft_consensus.cc:504] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } }
I20260807 20:12:48.704404 26872 leader_election.cc:288] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [CANDIDATE]: Term 1 election: Requested vote from peers 345d68f75ab3496b991b822920e9686c (127.23.200.194:45443), e1545a2943db4e799aa04fc4860b99d8 (127.23.200.193:42097)
I20260807 20:12:48.704530 26539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f8cc735be69460586cf64c4ea8cc222" candidate_uuid: "3fdf7e0f624f4f3496458a93316ebda9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "345d68f75ab3496b991b822920e9686c"
I20260807 20:12:48.704586 26318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f8cc735be69460586cf64c4ea8cc222" candidate_uuid: "3fdf7e0f624f4f3496458a93316ebda9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1545a2943db4e799aa04fc4860b99d8"
I20260807 20:12:48.704618 26539 raft_consensus.cc:3037] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:48.704653 26318 raft_consensus.cc:3037] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:48.705407 26318 raft_consensus.cc:2454] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fdf7e0f624f4f3496458a93316ebda9 in term 1.
I20260807 20:12:48.705444 26539 raft_consensus.cc:2454] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fdf7e0f624f4f3496458a93316ebda9 in term 1.
I20260807 20:12:48.705883 26653 leader_election.cc:302] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fdf7e0f624f4f3496458a93316ebda9, e1545a2943db4e799aa04fc4860b99d8; no voters:
I20260807 20:12:48.706003 26872 raft_consensus.cc:2781] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:48.706157 26872 raft_consensus.cc:686] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [term 1 LEADER]: Becoming Leader. State: Replica: 3fdf7e0f624f4f3496458a93316ebda9, State: Running, Role: LEADER
I20260807 20:12:48.706241 26872 consensus_queue.cc:237] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } }
I20260807 20:12:48.707087 26084 catalog_manager.cc:5495] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fdf7e0f624f4f3496458a93316ebda9 (127.23.200.195). New cstate: current_term: 1 leader_uuid: "3fdf7e0f624f4f3496458a93316ebda9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } health_report { overall_health: UNKNOWN } } }
I20260807 20:12:48.716523 26859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.714168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32848 (local address 127.23.200.195:36587)
0807 20:12:48.714210 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.714213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.714246 (+ 33us) server_negotiation.cc:407] Connection header received
0807 20:12:48.714333 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.714334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.714352 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.714435 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:48.714560 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.715009 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.715767 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.715953 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.716006 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.716198 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:48.716288 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:48.716353 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.716411 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.716434 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:12:48.718194 26539 raft_consensus.cc:1261] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c [term 1 FOLLOWER]: Refusing update from remote peer 3fdf7e0f624f4f3496458a93316ebda9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:48.718204 26318 raft_consensus.cc:1261] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8 [term 1 FOLLOWER]: Refusing update from remote peer 3fdf7e0f624f4f3496458a93316ebda9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:48.718520 26872 consensus_queue.cc:1035] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:48.718652 26876 consensus_queue.cc:1035] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:48.720950 26890 mvcc.cc:205] Tried to move back new op lower bound from 7316003097467482112 to 7316003097421463552. Current Snapshot: MvccSnapshot[applied={T|T < 7316003097467482112}]
I20260807 20:12:48.721244 26885 mvcc.cc:205] Tried to move back new op lower bound from 7316003097467482112 to 7316003097421463552. Current Snapshot: MvccSnapshot[applied={T|T < 7316003097467482112 or (T in {7316003097467482112})}]
I20260807 20:12:48.724786 26084 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:48.724923 26084 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:48.725782 26084 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be: Sending DeleteTablet for 3 replicas of tablet 8f8cc735be69460586cf64c4ea8cc222
I20260807 20:12:48.726187 26729 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f8cc735be69460586cf64c4ea8cc222 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:48 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:32832
I20260807 20:12:48.726246 26508 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f8cc735be69460586cf64c4ea8cc222 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:48 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:55052
I20260807 20:12:48.726433 26903 tablet_replica.cc:330] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9: stopping tablet replica
I20260807 20:12:48.726434 26285 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f8cc735be69460586cf64c4ea8cc222 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:48 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:33098
I20260807 20:12:48.726526 26903 raft_consensus.cc:2229] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:12:48.726645 26903 raft_consensus.cc:2258] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:48.726691 26084 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:12:48.726830 26084 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:12:48.726899 26906 tablet_replica.cc:330] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8: stopping tablet replica
I20260807 20:12:48.726987 26906 raft_consensus.cc:2229] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:48.727101 26906 raft_consensus.cc:2258] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:48.727128 26903 ts_tablet_manager.cc:1905] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:48.727563 26906 ts_tablet_manager.cc:1905] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:48.728155 26508 tablet_service.cc:1467] Processing CreateTablet for tablet 54973791d5c54b189211a15cf178b558 (DEFAULT_TABLE table=test-table [id=619267027a4141989ef3855cea5bb821]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:48.728274 26508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54973791d5c54b189211a15cf178b558. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:48.729027 26903 ts_tablet_manager.cc:1918] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:12:48.729127 26903 log.cc:1196] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/8f8cc735be69460586cf64c4ea8cc222
I20260807 20:12:48.729161 26906 ts_tablet_manager.cc:1918] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:12:48.729233 26906 log.cc:1196] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/wal/wals/8f8cc735be69460586cf64c4ea8cc222
I20260807 20:12:48.729384 26903 ts_tablet_manager.cc:1939] T 8f8cc735be69460586cf64c4ea8cc222 P 3fdf7e0f624f4f3496458a93316ebda9: Deleting consensus metadata
I20260807 20:12:48.729485 26906 ts_tablet_manager.cc:1939] T 8f8cc735be69460586cf64c4ea8cc222 P e1545a2943db4e799aa04fc4860b99d8: Deleting consensus metadata
I20260807 20:12:48.729686 26866 tablet_bootstrap.cc:492] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c: Bootstrap starting.
I20260807 20:12:48.730028 26904 tablet_replica.cc:330] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c: stopping tablet replica
I20260807 20:12:48.730147 26904 raft_consensus.cc:2229] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:48.730254 26904 raft_consensus.cc:2258] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:48.730552 26866 tablet_bootstrap.cc:654] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:48.730728 26904 ts_tablet_manager.cc:1905] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:48.731153 26059 catalog_manager.cc:4841] TS e1545a2943db4e799aa04fc4860b99d8 (127.23.200.193:42097): tablet 8f8cc735be69460586cf64c4ea8cc222 (table test-table [id=801d0b878471496791353e9bd66575c7]) successfully deleted
I20260807 20:12:48.731185 26866 tablet_bootstrap.cc:492] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c: No bootstrap required, opened a new log
I20260807 20:12:48.731230 26059 catalog_manager.cc:4841] TS 3fdf7e0f624f4f3496458a93316ebda9 (127.23.200.195:36587): tablet 8f8cc735be69460586cf64c4ea8cc222 (table test-table [id=801d0b878471496791353e9bd66575c7]) successfully deleted
I20260807 20:12:48.731228 26866 ts_tablet_manager.cc:1397] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260807 20:12:48.731364 26866 raft_consensus.cc:348] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:48.731402 26285 tablet_service.cc:1467] Processing CreateTablet for tablet 54973791d5c54b189211a15cf178b558 (DEFAULT_TABLE table=test-table [id=619267027a4141989ef3855cea5bb821]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:48.731429 26866 raft_consensus.cc:374] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:48.731452 26866 raft_consensus.cc:729] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345d68f75ab3496b991b822920e9686c, State: Initialized, Role: FOLLOWER
I20260807 20:12:48.731513 26285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54973791d5c54b189211a15cf178b558. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:48.731495 26866 consensus_queue.cc:260] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:48.731578 26866 ts_tablet_manager.cc:1428] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:48.732599 26865 tablet_bootstrap.cc:492] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8: Bootstrap starting.
I20260807 20:12:48.732645 26729 tablet_service.cc:1467] Processing CreateTablet for tablet 54973791d5c54b189211a15cf178b558 (DEFAULT_TABLE table=test-table [id=619267027a4141989ef3855cea5bb821]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:48.732754 26729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54973791d5c54b189211a15cf178b558. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:48.733016 26865 tablet_bootstrap.cc:654] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:48.733690 26865 tablet_bootstrap.cc:492] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8: No bootstrap required, opened a new log
I20260807 20:12:48.733737 26865 ts_tablet_manager.cc:1397] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:48.733898 26864 tablet_bootstrap.cc:492] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9: Bootstrap starting.
I20260807 20:12:48.733872 26865 raft_consensus.cc:348] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:48.733942 26865 raft_consensus.cc:374] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:48.733961 26865 raft_consensus.cc:729] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1545a2943db4e799aa04fc4860b99d8, State: Initialized, Role: FOLLOWER
I20260807 20:12:48.733999 26865 consensus_queue.cc:260] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:48.734094 26865 ts_tablet_manager.cc:1428] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:48.734336 26864 tablet_bootstrap.cc:654] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:48.734973 26904 ts_tablet_manager.cc:1918] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:12:48.735004 26864 tablet_bootstrap.cc:492] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9: No bootstrap required, opened a new log
I20260807 20:12:48.735038 26904 log.cc:1196] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/8f8cc735be69460586cf64c4ea8cc222
I20260807 20:12:48.735052 26864 ts_tablet_manager.cc:1397] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:48.735180 26864 raft_consensus.cc:348] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:48.735248 26864 raft_consensus.cc:374] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:48.735273 26864 raft_consensus.cc:729] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fdf7e0f624f4f3496458a93316ebda9, State: Initialized, Role: FOLLOWER
I20260807 20:12:48.735261 26904 ts_tablet_manager.cc:1939] T 8f8cc735be69460586cf64c4ea8cc222 P 345d68f75ab3496b991b822920e9686c: Deleting consensus metadata
I20260807 20:12:48.735318 26864 consensus_queue.cc:260] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:48.735397 26864 ts_tablet_manager.cc:1428] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:48.735685 26061 catalog_manager.cc:4841] TS 345d68f75ab3496b991b822920e9686c (127.23.200.194:45443): tablet 8f8cc735be69460586cf64c4ea8cc222 (table test-table [id=801d0b878471496791353e9bd66575c7]) successfully deleted
W20260807 20:12:48.777657 26395 tablet.cc:2369] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:12:48.834024 26828 tablet.cc:2369] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:12:48.862632 26874 raft_consensus.cc:482] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:12:48.862735 26874 raft_consensus.cc:504] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:48.863132 26874 leader_election.cc:288] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1545a2943db4e799aa04fc4860b99d8 (127.23.200.193:42097), 3fdf7e0f624f4f3496458a93316ebda9 (127.23.200.195:36587)
I20260807 20:12:48.867168 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.863332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:42097 (local address 127.23.200.194:44629)
0807 20:12:48.863414 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:48.863432 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:48.863538 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:48.863803 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:48.863804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:48.863806 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:48.863885 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.863887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.864852 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.864853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.866088 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.866092 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.866866 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.866867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.866961 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.866966 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:12:48.867024 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 20:12:48.867062 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260807 20:12:48.867399 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.863387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:44629 (local address 127.23.200.193:42097)
0807 20:12:48.863445 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.863447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.863517 (+ 70us) server_negotiation.cc:407] Connection header received
0807 20:12:48.863640 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.863642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.863653 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.863709 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:48.863958 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.864750 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.866166 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.866733 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.866802 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.866812 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.867283 (+ 471us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.867340 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:12:48.867393 26859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.863560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:48479 (local address 127.23.200.195:36587)
0807 20:12:48.863610 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.863612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.863676 (+ 64us) server_negotiation.cc:407] Connection header received
0807 20:12:48.863752 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.863753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.863765 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.863806 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:48.864023 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.864747 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.866402 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.866995 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.867062 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.867070 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.867270 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.867321 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:12:48.867643 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.863444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:36587 (local address 127.23.200.194:48479)
0807 20:12:48.863620 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:48.863630 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:48.863692 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:48.863828 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:48.863829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:48.863831 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:48.863950 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.863952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.865234 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.865235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.866298 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.866302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.867106 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.867107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.867187 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.867192 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:12:48.867510 (+ 318us) client_negotiation.cc:240] Negotiation successful
0807 20:12:48.867554 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":59,"threads_started":1}
I20260807 20:12:48.867677 26318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54973791d5c54b189211a15cf178b558" candidate_uuid: "345d68f75ab3496b991b822920e9686c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1545a2943db4e799aa04fc4860b99d8" is_pre_election: true
I20260807 20:12:48.867784 26318 raft_consensus.cc:2454] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 345d68f75ab3496b991b822920e9686c in term 0.
I20260807 20:12:48.868024 26434 leader_election.cc:302] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 345d68f75ab3496b991b822920e9686c, e1545a2943db4e799aa04fc4860b99d8; no voters:
I20260807 20:12:48.868220 26874 raft_consensus.cc:2781] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:12:48.868259 26874 raft_consensus.cc:482] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:12:48.868281 26874 raft_consensus.cc:3037] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:48.868373 26758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54973791d5c54b189211a15cf178b558" candidate_uuid: "345d68f75ab3496b991b822920e9686c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fdf7e0f624f4f3496458a93316ebda9" is_pre_election: true
I20260807 20:12:48.868511 26758 raft_consensus.cc:2454] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 345d68f75ab3496b991b822920e9686c in term 0.
I20260807 20:12:48.869223 26874 raft_consensus.cc:504] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:48.869365 26874 leader_election.cc:288] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [CANDIDATE]: Term 1 election: Requested vote from peers e1545a2943db4e799aa04fc4860b99d8 (127.23.200.193:42097), 3fdf7e0f624f4f3496458a93316ebda9 (127.23.200.195:36587)
I20260807 20:12:48.869596 26318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54973791d5c54b189211a15cf178b558" candidate_uuid: "345d68f75ab3496b991b822920e9686c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1545a2943db4e799aa04fc4860b99d8"
I20260807 20:12:48.869608 26758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54973791d5c54b189211a15cf178b558" candidate_uuid: "345d68f75ab3496b991b822920e9686c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fdf7e0f624f4f3496458a93316ebda9"
I20260807 20:12:48.869673 26758 raft_consensus.cc:3037] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:48.869673 26318 raft_consensus.cc:3037] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:48.870402 26318 raft_consensus.cc:2454] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 345d68f75ab3496b991b822920e9686c in term 1.
I20260807 20:12:48.870447 26758 raft_consensus.cc:2454] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 345d68f75ab3496b991b822920e9686c in term 1.
I20260807 20:12:48.870586 26434 leader_election.cc:302] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 345d68f75ab3496b991b822920e9686c, 3fdf7e0f624f4f3496458a93316ebda9; no voters:
I20260807 20:12:48.870738 26874 raft_consensus.cc:2781] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:48.870883 26874 raft_consensus.cc:686] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [term 1 LEADER]: Becoming Leader. State: Replica: 345d68f75ab3496b991b822920e9686c, State: Running, Role: LEADER
I20260807 20:12:48.870947 26874 consensus_queue.cc:237] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:48.871685 26083 catalog_manager.cc:5495] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c reported cstate change: term changed from 0 to 1, leader changed from <none> to 345d68f75ab3496b991b822920e9686c (127.23.200.194). New cstate: current_term: 1 leader_uuid: "345d68f75ab3496b991b822920e9686c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } health_report { overall_health: UNKNOWN } } }
Aug 07 20:12:48 dist-test-slave-2j7r krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133568, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:48.912253 26923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.905792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:36021 (local address 127.0.0.1:38522)
0807 20:12:48.905976 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:48.905985 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:48.906059 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:48.906179 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:48.906180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:48.906508 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:48.906557 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.906560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.907206 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.907207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.907777 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:48.907780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.908083 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.908084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:48.908133 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.908613 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:48.908618 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:48.909987 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:48.911261 (+ 1274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:48.911269 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:48.911272 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:48.911464 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:48.911629 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:48.911630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:48.911630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:48.911684 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:48.911886 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:48.911901 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:48.911948 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 20:12:48.912007 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 20:12:48.912127 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":103,"threads_started":1}
I20260807 20:12:48.912253 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:48.905836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38522 (local address 127.23.200.254:36021)
0807 20:12:48.905898 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:48.905900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:48.906048 (+ 148us) server_negotiation.cc:407] Connection header received
0807 20:12:48.906111 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:48.906111 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:48.906122 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:48.906170 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:48.906635 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.907134 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.907848 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:48.908027 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:48.908081 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:48.910063 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:48.910065 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:48.910065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:48.910072 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:48.911191 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.911528 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.911529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.911530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.911578 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:48.911733 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:48.911733 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:48.911734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:48.911834 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:48.911888 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:48.912036 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 20:12:48.912176 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":36}
I20260807 20:12:48.915294 26083 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:38522:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 20:12:48.915524 26083 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:12:48.917008 26508 tablet_service.cc:1467] Processing CreateTablet for tablet 7f903e506ca94296887ac3c15e4c57c3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=963b83b8636b4528a832b68898e46f55]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:12:48.917029 26729 tablet_service.cc:1467] Processing CreateTablet for tablet 7f903e506ca94296887ac3c15e4c57c3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=963b83b8636b4528a832b68898e46f55]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:12:48.917210 26508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f903e506ca94296887ac3c15e4c57c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:48.917212 26729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f903e506ca94296887ac3c15e4c57c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:48.918725 26864 tablet_bootstrap.cc:492] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9: Bootstrap starting.
I20260807 20:12:48.918833 26285 tablet_service.cc:1467] Processing CreateTablet for tablet 7f903e506ca94296887ac3c15e4c57c3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=963b83b8636b4528a832b68898e46f55]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:12:48.918972 26285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f903e506ca94296887ac3c15e4c57c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:48.919090 26864 tablet_bootstrap.cc:654] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:48.919354 26866 tablet_bootstrap.cc:492] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c: Bootstrap starting.
I20260807 20:12:48.919690 26864 tablet_bootstrap.cc:492] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9: No bootstrap required, opened a new log
I20260807 20:12:48.919744 26864 ts_tablet_manager.cc:1397] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:48.919788 26866 tablet_bootstrap.cc:654] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:48.919858 26864 raft_consensus.cc:348] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:48.919961 26864 raft_consensus.cc:374] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:48.919981 26864 raft_consensus.cc:729] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fdf7e0f624f4f3496458a93316ebda9, State: Initialized, Role: FOLLOWER
I20260807 20:12:48.920029 26864 consensus_queue.cc:260] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:48.920141 26864 ts_tablet_manager.cc:1428] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:48.920173 26876 tablet_replica.cc:439] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } }
I20260807 20:12:48.920249 26876 tablet_replica.cc:442] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:12:48.920322 26865 tablet_bootstrap.cc:492] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8: Bootstrap starting.
I20260807 20:12:48.920660 26866 tablet_bootstrap.cc:492] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c: No bootstrap required, opened a new log
I20260807 20:12:48.920701 26866 ts_tablet_manager.cc:1397] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260807 20:12:48.920717 26865 tablet_bootstrap.cc:654] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:48.920835 26866 raft_consensus.cc:348] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:48.920893 26866 raft_consensus.cc:374] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:48.920910 26866 raft_consensus.cc:729] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345d68f75ab3496b991b822920e9686c, State: Initialized, Role: FOLLOWER
I20260807 20:12:48.920956 26866 consensus_queue.cc:260] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:48.921042 26866 ts_tablet_manager.cc:1428] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:48.921185 26874 tablet_replica.cc:439] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } }
I20260807 20:12:48.921262 26874 tablet_replica.cc:442] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:12:48.921504 26865 tablet_bootstrap.cc:492] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8: No bootstrap required, opened a new log
I20260807 20:12:48.921556 26865 ts_tablet_manager.cc:1397] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:48.921707 26865 raft_consensus.cc:348] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:48.921782 26865 raft_consensus.cc:374] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:48.921794 26865 raft_consensus.cc:729] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1545a2943db4e799aa04fc4860b99d8, State: Initialized, Role: FOLLOWER
I20260807 20:12:48.921840 26865 consensus_queue.cc:260] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:48.921928 26865 ts_tablet_manager.cc:1428] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:48.921993 26873 tablet_replica.cc:439] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } }
I20260807 20:12:48.922087 26873 tablet_replica.cc:442] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:12:49.177706 26874 raft_consensus.cc:482] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:12:49.177839 26874 raft_consensus.cc:504] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:49.177977 26874 leader_election.cc:288] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1545a2943db4e799aa04fc4860b99d8 (127.23.200.193:42097), 3fdf7e0f624f4f3496458a93316ebda9 (127.23.200.195:36587)
I20260807 20:12:49.178325 26318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f903e506ca94296887ac3c15e4c57c3" candidate_uuid: "345d68f75ab3496b991b822920e9686c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1545a2943db4e799aa04fc4860b99d8" is_pre_election: true
I20260807 20:12:49.178328 26758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f903e506ca94296887ac3c15e4c57c3" candidate_uuid: "345d68f75ab3496b991b822920e9686c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fdf7e0f624f4f3496458a93316ebda9" is_pre_election: true
I20260807 20:12:49.178438 26318 raft_consensus.cc:2454] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 345d68f75ab3496b991b822920e9686c in term 0.
I20260807 20:12:49.178450 26758 raft_consensus.cc:2454] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 345d68f75ab3496b991b822920e9686c in term 0.
I20260807 20:12:49.178611 26434 leader_election.cc:302] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 345d68f75ab3496b991b822920e9686c, e1545a2943db4e799aa04fc4860b99d8; no voters:
I20260807 20:12:49.178736 26874 raft_consensus.cc:2781] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:12:49.178792 26874 raft_consensus.cc:482] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:12:49.178819 26874 raft_consensus.cc:3037] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:49.179628 26874 raft_consensus.cc:504] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:49.179787 26874 leader_election.cc:288] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [CANDIDATE]: Term 1 election: Requested vote from peers e1545a2943db4e799aa04fc4860b99d8 (127.23.200.193:42097), 3fdf7e0f624f4f3496458a93316ebda9 (127.23.200.195:36587)
I20260807 20:12:49.179993 26318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f903e506ca94296887ac3c15e4c57c3" candidate_uuid: "345d68f75ab3496b991b822920e9686c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1545a2943db4e799aa04fc4860b99d8"
I20260807 20:12:49.179983 26758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f903e506ca94296887ac3c15e4c57c3" candidate_uuid: "345d68f75ab3496b991b822920e9686c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fdf7e0f624f4f3496458a93316ebda9"
I20260807 20:12:49.180059 26318 raft_consensus.cc:3037] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:49.180068 26758 raft_consensus.cc:3037] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:49.180708 26318 raft_consensus.cc:2454] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 345d68f75ab3496b991b822920e9686c in term 1.
I20260807 20:12:49.180763 26758 raft_consensus.cc:2454] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 345d68f75ab3496b991b822920e9686c in term 1.
I20260807 20:12:49.180892 26434 leader_election.cc:302] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 345d68f75ab3496b991b822920e9686c, e1545a2943db4e799aa04fc4860b99d8; no voters:
I20260807 20:12:49.180986 26874 raft_consensus.cc:2781] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:49.181041 26874 raft_consensus.cc:686] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [term 1 LEADER]: Becoming Leader. State: Replica: 345d68f75ab3496b991b822920e9686c, State: Running, Role: LEADER
I20260807 20:12:49.181120 26874 consensus_queue.cc:237] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } }
I20260807 20:12:49.181459 26913 tablet_replica.cc:439] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c: TxnStatusTablet state changed. Reason: New leader 345d68f75ab3496b991b822920e9686c. Latest consensus state: current_term: 1 leader_uuid: "345d68f75ab3496b991b822920e9686c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } }
I20260807 20:12:49.181550 26913 tablet_replica.cc:442] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:12:49.181725 26928 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 20:12:49.181844 26928 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 20:12:49.181777 26083 catalog_manager.cc:5495] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c reported cstate change: term changed from 0 to 1, leader changed from <none> to 345d68f75ab3496b991b822920e9686c (127.23.200.194). New cstate: current_term: 1 leader_uuid: "345d68f75ab3496b991b822920e9686c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } health_report { overall_health: UNKNOWN } } }
I20260807 20:12:49.205920 26931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:49.203443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55064 (local address 127.23.200.194:45443)
0807 20:12:49.203559 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:49.203561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:49.203569 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:12:49.203608 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:49.203609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:49.203630 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:49.203697 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:49.203829 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:49.204337 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:49.205051 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:49.205329 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:49.205387 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:49.205564 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:49.205675 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:49.205738 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:49.205787 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 20:12:49.205816 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":63,"threads_started":1}
I20260807 20:12:49.205907 26923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:49.203347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:45443 (local address 127.0.0.1:55064)
0807 20:12:49.203428 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:49.203442 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:49.203506 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:49.203709 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:49.203710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:49.203711 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:12:49.203766 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:49.203768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:49.204411 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:49.204412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:49.204974 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:49.204977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:49.205395 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:49.205397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:49.205459 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:49.205476 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:12:49.205732 (+ 256us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:12:49.205740 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 20:12:49.205784 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 20:12:49.205835 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 20:12:49.207238 26318 raft_consensus.cc:1261] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8 [term 1 FOLLOWER]: Refusing update from remote peer 345d68f75ab3496b991b822920e9686c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:49.207289 26758 raft_consensus.cc:1261] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9 [term 1 FOLLOWER]: Refusing update from remote peer 345d68f75ab3496b991b822920e9686c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:49.207653 26913 consensus_queue.cc:1035] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:49.207774 26874 consensus_queue.cc:1035] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:49.208338 26873 tablet_replica.cc:439] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8: TxnStatusTablet state changed. Reason: New leader 345d68f75ab3496b991b822920e9686c. Latest consensus state: current_term: 1 leader_uuid: "345d68f75ab3496b991b822920e9686c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } }
I20260807 20:12:49.208436 26873 tablet_replica.cc:442] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:12:49.208532 26876 tablet_replica.cc:439] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9: TxnStatusTablet state changed. Reason: New leader 345d68f75ab3496b991b822920e9686c. Latest consensus state: current_term: 1 leader_uuid: "345d68f75ab3496b991b822920e9686c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } }
I20260807 20:12:49.208621 26876 tablet_replica.cc:442] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:12:49.209892 26874 tablet_replica.cc:439] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "345d68f75ab3496b991b822920e9686c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } }
I20260807 20:12:49.209997 26874 tablet_replica.cc:442] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:12:49.210019 26889 mvcc.cc:205] Tried to move back new op lower bound from 7316003099470692352 to 7316003099366363136. Current Snapshot: MvccSnapshot[applied={T|T < 7316003099470692352}]
I20260807 20:12:49.209942 26913 tablet_replica.cc:439] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "345d68f75ab3496b991b822920e9686c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } }
I20260807 20:12:49.210366 26913 tablet_replica.cc:442] T 7f903e506ca94296887ac3c15e4c57c3 P 345d68f75ab3496b991b822920e9686c: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:12:49.210729 26873 tablet_replica.cc:439] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "345d68f75ab3496b991b822920e9686c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } }
I20260807 20:12:49.210816 26873 tablet_replica.cc:442] T 7f903e506ca94296887ac3c15e4c57c3 P e1545a2943db4e799aa04fc4860b99d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:12:49.211486 26876 tablet_replica.cc:439] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "345d68f75ab3496b991b822920e9686c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 } } peers { permanent_uuid: "345d68f75ab3496b991b822920e9686c" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45443 } } peers { permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 } } }
I20260807 20:12:49.211571 26876 tablet_replica.cc:442] T 7f903e506ca94296887ac3c15e4c57c3 P 3fdf7e0f624f4f3496458a93316ebda9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:12:49.214423 26931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:49.212273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55072 (local address 127.23.200.194:45443)
0807 20:12:49.212422 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:49.212424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:49.212429 (+ 5us) server_negotiation.cc:407] Connection header received
0807 20:12:49.212444 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:49.212445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:49.212453 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:49.212492 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:49.212614 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:49.213145 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:49.213841 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:49.214031 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:49.214084 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:49.214209 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:49.214243 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:49.214285 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:49.214333 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 20:12:49.214356 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260807 20:12:49.219273 26931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:49.217023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:36371 (local address 127.23.200.194:45443)
0807 20:12:49.217082 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:49.217102 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:49.217123 (+ 21us) server_negotiation.cc:407] Connection header received
0807 20:12:49.217186 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:49.217186 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:49.217195 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:49.217243 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:49.217386 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:49.217904 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:49.218647 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:49.218838 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:49.218896 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:49.219053 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:49.219083 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:49.219124 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:49.219175 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 20:12:49.219191 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 20:12:49.219292 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:49.216863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:45443 (local address 127.23.200.194:36371)
0807 20:12:49.217054 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:49.217065 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:49.217139 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:49.217246 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:49.217247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:49.217248 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:12:49.217333 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:49.217335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:49.217973 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:49.217973 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:49.218573 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:49.218575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:49.218892 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:49.218892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:49.218970 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:49.218986 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:12:49.219124 (+ 138us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:12:49.219126 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:12:49.219171 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 20:12:49.219207 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
I20260807 20:12:49.221812 26318 raft_consensus.cc:1261] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8 [term 1 FOLLOWER]: Refusing update from remote peer 345d68f75ab3496b991b822920e9686c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:49.221813 26758 raft_consensus.cc:1261] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9 [term 1 FOLLOWER]: Refusing update from remote peer 345d68f75ab3496b991b822920e9686c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:49.222084 26937 consensus_queue.cc:1035] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1545a2943db4e799aa04fc4860b99d8" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:49.222203 26913 consensus_queue.cc:1035] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fdf7e0f624f4f3496458a93316ebda9" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:49.234663 26084 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:49.234786 26084 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:49.235383 26084 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81acf40261e545e7b50ab33e40b9f4be: Sending DeleteTablet for 3 replicas of tablet 54973791d5c54b189211a15cf178b558
I20260807 20:12:49.235644 26508 tablet_service.cc:1514] Processing DeleteTablet for tablet 54973791d5c54b189211a15cf178b558 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:49 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:55052
I20260807 20:12:49.235693 26285 tablet_service.cc:1514] Processing DeleteTablet for tablet 54973791d5c54b189211a15cf178b558 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:49 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:33098
I20260807 20:12:49.235721 26729 tablet_service.cc:1514] Processing DeleteTablet for tablet 54973791d5c54b189211a15cf178b558 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:49 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:32832
I20260807 20:12:49.235867 26904 tablet_replica.cc:330] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c: stopping tablet replica
I20260807 20:12:49.235922 26904 raft_consensus.cc:2229] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:12:49.235997 26904 raft_consensus.cc:2258] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:49.236222 26904 ts_tablet_manager.cc:1905] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:49.236364 26949 tablet_replica.cc:330] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8: stopping tablet replica
I20260807 20:12:49.236435 26949 raft_consensus.cc:2229] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:49.236477 26949 raft_consensus.cc:2258] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:49.236721 26949 ts_tablet_manager.cc:1905] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:49.237187 26948 tablet_replica.cc:330] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9: stopping tablet replica
I20260807 20:12:49.237277 26948 raft_consensus.cc:2229] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:49.237329 26948 raft_consensus.cc:2258] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:49.237581 26948 ts_tablet_manager.cc:1905] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:49.237834 26904 ts_tablet_manager.cc:1918] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:12:49.237895 26904 log.cc:1196] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/54973791d5c54b189211a15cf178b558
I20260807 20:12:49.238090 26904 ts_tablet_manager.cc:1939] T 54973791d5c54b189211a15cf178b558 P 345d68f75ab3496b991b822920e9686c: Deleting consensus metadata
I20260807 20:12:49.238410 26949 ts_tablet_manager.cc:1918] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:12:49.238473 26949 log.cc:1196] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-0/wal/wals/54973791d5c54b189211a15cf178b558
I20260807 20:12:49.238760 26061 catalog_manager.cc:4841] TS 345d68f75ab3496b991b822920e9686c (127.23.200.194:45443): tablet 54973791d5c54b189211a15cf178b558 (table test-table [id=619267027a4141989ef3855cea5bb821]) successfully deleted
I20260807 20:12:49.238922 26949 ts_tablet_manager.cc:1939] T 54973791d5c54b189211a15cf178b558 P e1545a2943db4e799aa04fc4860b99d8: Deleting consensus metadata
I20260807 20:12:49.239218 26059 catalog_manager.cc:4841] TS e1545a2943db4e799aa04fc4860b99d8 (127.23.200.193:42097): tablet 54973791d5c54b189211a15cf178b558 (table test-table [id=619267027a4141989ef3855cea5bb821]) successfully deleted
I20260807 20:12:49.239534 26948 ts_tablet_manager.cc:1918] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:12:49.239631 26948 log.cc:1196] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/54973791d5c54b189211a15cf178b558
I20260807 20:12:49.239864 26948 ts_tablet_manager.cc:1939] T 54973791d5c54b189211a15cf178b558 P 3fdf7e0f624f4f3496458a93316ebda9: Deleting consensus metadata
I20260807 20:12:49.240168 26059 catalog_manager.cc:4841] TS 3fdf7e0f624f4f3496458a93316ebda9 (127.23.200.195:36587): tablet 54973791d5c54b189211a15cf178b558 (table test-table [id=619267027a4141989ef3855cea5bb821]) successfully deleted
Aug 07 20:12:49 dist-test-slave-2j7r krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133568, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.200.193@KRBTEST.COM
I20260807 20:12:49.247836 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:49.238231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33114 (local address 127.23.200.193:42097)
0807 20:12:49.238284 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:49.238287 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:49.238846 (+ 559us) server_negotiation.cc:407] Connection header received
0807 20:12:49.238975 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:49.238976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:49.238987 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:49.239015 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:49.239872 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:49.240625 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:49.241418 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:49.241713 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:49.241785 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:49.244733 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:49.244745 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:49.244745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:49.244754 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:49.246706 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:49.247047 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:49.247051 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:49.247052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:49.247137 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:49.247304 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:49.247304 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:49.247305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:49.247477 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:49.247618 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:49.247655 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 20:12:49.247738 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260807 20:12:49.248104 26383 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33114
I20260807 20:12:49.255765 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:49.249100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38528 (local address 127.23.200.254:36021)
0807 20:12:49.249146 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:49.249148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:49.249340 (+ 192us) server_negotiation.cc:407] Connection header received
0807 20:12:49.249429 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:49.249430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:49.249440 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:49.249484 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:49.250056 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:49.250561 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:49.251530 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:49.251718 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:49.251771 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:49.253051 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:49.253053 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:49.253053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:49.253060 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:49.254524 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:49.254879 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:49.254880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:49.254880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:49.254936 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:49.255152 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:49.255153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:49.255154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:49.255341 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:49.255440 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:49.255624 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 20:12:49.255695 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 20:12:49.256052 26084 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38528
I20260807 20:12:49.256677 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 26193
I20260807 20:12:49.270100 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 26408
I20260807 20:12:49.282579 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 26626
I20260807 20:12:49.299266 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 26037
2026-08-07T20:12:49Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2663 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:49 dist-test-slave-2j7r krb5kdc[26956](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:49 dist-test-slave-2j7r krb5kdc[26956](info): set up 2 sockets
Aug 07 20:12:49 dist-test-slave-2j7r krb5kdc[26956](info): commencing operation
krb5kdc: starting...
W20260807 20:12:50.465590 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:12:50 dist-test-slave-2j7r krb5kdc[26956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:12:50.585856 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:38575
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:41425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:38575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:50.669401 26980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:50.669633 26980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:50.669667 26980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:50.670831 26980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:50.670874 26980 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 20:12:50.670892 26980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:50.670908 26980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:50.670923 26980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:50.670940 26980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:50.672467 26980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:41425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:38575
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:38575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:50.672712 26980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:50.672937 26980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:50.677834 26990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:50.677842 26992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:50.678085 26997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:50.678079 26980 server_base.cc:1034] running on GCE node
I20260807 20:12:50.678371 26980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:50.678712 26980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:50.679875 26980 hybrid_clock.cc:648] HybridClock initialized: now 1786133570679847 us; error 42 us; skew 500 ppm
Aug 07 20:12:50 dist-test-slave-2j7r krb5kdc[26956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133570, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:50.682479 26980 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:12:50.682893 26980 webserver.cc:460] Webserver started at http://127.23.200.254:37769/ using document root <none> and password file <none>
I20260807 20:12:50.683104 26980 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:50.683144 26980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:50.683241 26980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:50.684281 26980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "cfd1e741b37c47e3923ca33f6e04a5d5"
format_stamp: "Formatted at 2026-08-07 20:12:50 on dist-test-slave-2j7r"
server_key: "1439926eb283999ca32f7c404c2c8fc3"
server_key_iv: "247a11ba4b3781da4fc4c9cdacdee955"
server_key_version: "encryptionkey@0"
I20260807 20:12:50.684576 26980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "cfd1e741b37c47e3923ca33f6e04a5d5"
format_stamp: "Formatted at 2026-08-07 20:12:50 on dist-test-slave-2j7r"
server_key: "1439926eb283999ca32f7c404c2c8fc3"
server_key_iv: "247a11ba4b3781da4fc4c9cdacdee955"
server_key_version: "encryptionkey@0"
I20260807 20:12:50.685876 26980 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:50.686573 27006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:50.686748 26980 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:50.686834 26980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "cfd1e741b37c47e3923ca33f6e04a5d5"
format_stamp: "Formatted at 2026-08-07 20:12:50 on dist-test-slave-2j7r"
server_key: "1439926eb283999ca32f7c404c2c8fc3"
server_key_iv: "247a11ba4b3781da4fc4c9cdacdee955"
server_key_version: "encryptionkey@0"
I20260807 20:12:50.686897 26980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:50.698827 26980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:50.700845 26980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:50.700963 26980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:50.705402 26980 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:38575
I20260807 20:12:50.705511 27101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:38575 every 8 connection(s)
I20260807 20:12:50.705789 26980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:50.706496 27103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:50.708928 27103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5: Bootstrap starting.
I20260807 20:12:50.709564 27103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:50.709832 27103 log.cc:824] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:50.710593 27103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5: No bootstrap required, opened a new log
I20260807 20:12:50.711000 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 26980
I20260807 20:12:50.711094 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:12:50.711256 24355 external_mini_cluster.cc:1428] Setting key 3e13b84498a9b3b68905566a6606a5e9
I20260807 20:12:50.713047 27103 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd1e741b37c47e3923ca33f6e04a5d5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38575 } }
I20260807 20:12:50.713225 27103 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:50.713267 27103 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd1e741b37c47e3923ca33f6e04a5d5, State: Initialized, Role: FOLLOWER
I20260807 20:12:50.713452 27103 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd1e741b37c47e3923ca33f6e04a5d5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38575 } }
I20260807 20:12:50.713533 27103 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:50.713567 27103 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:50.713616 27103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:50.714121 27103 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd1e741b37c47e3923ca33f6e04a5d5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38575 } }
I20260807 20:12:50.714218 27103 leader_election.cc:302] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfd1e741b37c47e3923ca33f6e04a5d5; no voters:
I20260807 20:12:50.714389 27103 leader_election.cc:288] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:50.714429 27111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:50.714536 27111 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [term 1 LEADER]: Becoming Leader. State: Replica: cfd1e741b37c47e3923ca33f6e04a5d5, State: Running, Role: LEADER
I20260807 20:12:50.714620 27111 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd1e741b37c47e3923ca33f6e04a5d5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38575 } }
I20260807 20:12:50.714653 27103 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:50.715008 27111 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfd1e741b37c47e3923ca33f6e04a5d5. Latest consensus state: current_term: 1 leader_uuid: "cfd1e741b37c47e3923ca33f6e04a5d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd1e741b37c47e3923ca33f6e04a5d5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38575 } } }
I20260807 20:12:50.715015 27113 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfd1e741b37c47e3923ca33f6e04a5d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd1e741b37c47e3923ca33f6e04a5d5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38575 } } }
I20260807 20:12:50.715090 27111 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:50.715123 27113 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5 [sys.catalog]: This master's current role is: LEADER
W20260807 20:12:50.716348 27137 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:12:50.716393 27137 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:12:50.716450 27119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:50.716616 27119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:50.718144 27119 catalog_manager.cc:1347] Generated new cluster ID: 33e8d476304d4337bfea03cf1d6acac3
I20260807 20:12:50.718189 27119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:12:50 dist-test-slave-2j7r krb5kdc[26956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:50.723738 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:50.711768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35470 (local address 127.23.200.254:38575)
0807 20:12:50.711989 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:50.711999 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:50.712026 (+ 27us) server_negotiation.cc:407] Connection header received
0807 20:12:50.712970 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:50.712978 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:50.713029 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:50.713207 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:50.713856 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:50.714977 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:50.716079 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:50.716611 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:50.716710 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:50.719498 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:50.719503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:50.719503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:50.719515 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:50.722066 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:50.722494 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:50.722497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:50.722498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:50.722599 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:50.722796 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:50.722797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:50.722797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:50.723085 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:50.723228 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:50.723366 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 20:12:50.723510 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":96,"threads_started":1}
I20260807 20:12:50.726449 27119 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:50.726917 27119 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:50.731010 27119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5: Generated new TSK 0
I20260807 20:12:50.731169 27119 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:12:50.758147 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:38575
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:41425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:50.840458 27143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:50.840664 27143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:50.840710 27143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:50.841863 27143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:50.841903 27143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:50.841930 27143 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 20:12:50.841945 27143 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:12:50.841969 27143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:12:50.843443 27143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:41425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.200.254:38575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:50.843670 27143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:50.843884 27143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:50.848773 27163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:50.848795 27158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:50.848770 27157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:50.849022 27143 server_base.cc:1034] running on GCE node
I20260807 20:12:50.849272 27143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:50.849502 27143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:50.850664 27143 hybrid_clock.cc:648] HybridClock initialized: now 1786133570850646 us; error 29 us; skew 500 ppm
Aug 07 20:12:50 dist-test-slave-2j7r krb5kdc[26956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133570, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:50.853332 27143 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:12:50.853883 27143 webserver.cc:460] Webserver started at http://127.23.200.193:41201/ using document root <none> and password file <none>
I20260807 20:12:50.854099 27143 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:50.854147 27143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:50.854276 27143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:50.855337 27143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b"
format_stamp: "Formatted at 2026-08-07 20:12:50 on dist-test-slave-2j7r"
server_key: "60a0640a6d3f179a5701932a56ce0651"
server_key_iv: "b5bb48caf80945081e2ec9537f877b4c"
server_key_version: "encryptionkey@0"
I20260807 20:12:50.855675 27143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b"
format_stamp: "Formatted at 2026-08-07 20:12:50 on dist-test-slave-2j7r"
server_key: "60a0640a6d3f179a5701932a56ce0651"
server_key_iv: "b5bb48caf80945081e2ec9537f877b4c"
server_key_version: "encryptionkey@0"
I20260807 20:12:50.856902 27143 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:12:50.857645 27174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:50.857805 27143 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:50.857890 27143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b"
format_stamp: "Formatted at 2026-08-07 20:12:50 on dist-test-slave-2j7r"
server_key: "60a0640a6d3f179a5701932a56ce0651"
server_key_iv: "b5bb48caf80945081e2ec9537f877b4c"
server_key_version: "encryptionkey@0"
I20260807 20:12:50.857955 27143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:50.879659 27143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:50.881888 27143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:50.882036 27143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:50.882561 27143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:50.882603 27143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:50.882637 27143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:50.882661 27143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 20:12:50 dist-test-slave-2j7r krb5kdc[26956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133570, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:50.889802 27368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:46743 every 8 connection(s)
I20260807 20:12:50.889729 27143 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:46743
I20260807 20:12:50.890468 27143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:12:50.892645 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:50.883079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:45429 (local address 127.23.200.254:38575)
0807 20:12:50.883144 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:50.883147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:50.883376 (+ 229us) server_negotiation.cc:407] Connection header received
0807 20:12:50.883940 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:50.883942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:50.883956 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:50.884010 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:50.884948 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:50.885743 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:50.887238 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:50.887463 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:50.887514 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:50.890153 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:50.890155 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:50.890155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:50.890163 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:50.891491 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:50.891869 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:50.891871 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:50.891871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:50.891935 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:50.892145 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:50.892146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:50.892146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:50.892250 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:50.892303 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:50.892503 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 20:12:50.892571 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 20:12:50.892774 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:50.883077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:38575 (local address 127.23.200.193:45429)
0807 20:12:50.883242 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:50.883278 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:50.883832 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:50.884130 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:50.884136 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:50.884505 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:50.884817 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:50.884824 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:50.885852 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:50.885853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:50.887140 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:50.887143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:50.887549 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:50.887550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:50.887700 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:50.888207 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:50.888218 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:50.890054 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:50.891560 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:50.891563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:50.891565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:50.891763 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:50.892006 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:50.892006 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:50.892007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:50.892099 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:50.892293 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:50.892296 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:50.892406 (+ 110us) client_negotiation.cc:769] Sending connection context
0807 20:12:50.892481 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 20:12:50.892597 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":98,"threads_started":1}
I20260807 20:12:50.893353 27371 heartbeater.cc:347] Connected to a master server at 127.23.200.254:38575
I20260807 20:12:50.893451 27371 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:50.893658 27371 heartbeater.cc:511] Master 127.23.200.254:38575 requested a full tablet report, sending...
I20260807 20:12:50.893739 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 27143
I20260807 20:12:50.893831 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:12:50.894006 24355 external_mini_cluster.cc:1428] Setting key 4a8a4e2047153db07d2bb9007ce42c7b
I20260807 20:12:50.894510 27031 ts_manager.cc:194] Registered new tserver with Master: 8cd3bdfda33543cdb8582c3e4eb33c8b (127.23.200.193:46743)
I20260807 20:12:50.895344 27031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:45429
I20260807 20:12:50.899956 27382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:50.894422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:38575 (local address 127.23.200.193:51423)
0807 20:12:50.894638 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:50.894649 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:50.894704 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:50.894832 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:50.894833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:50.895022 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:50.895083 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:50.895085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:50.895847 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:50.895848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:50.896436 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:50.896439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:50.896761 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:50.896761 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:50.896832 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:50.897225 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:50.897230 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:50.897583 (+ 353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:50.899007 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:50.899007 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:50.899008 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:50.899162 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:50.899446 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:50.899447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:50.899447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:50.899499 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:50.899732 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:50.899733 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:50.899766 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 20:12:50.899812 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 20:12:50.899883 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":75,"threads_started":1}
I20260807 20:12:50.900003 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:50.894356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:51423 (local address 127.23.200.254:38575)
0807 20:12:50.894391 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:50.894392 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:50.894697 (+ 305us) server_negotiation.cc:407] Connection header received
0807 20:12:50.894750 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:50.894750 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:50.894761 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:50.894825 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:50.895150 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:50.895759 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:50.896514 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:50.896699 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:50.896752 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:50.897666 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:50.897667 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:50.897668 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:50.897676 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:50.898927 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:50.899237 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:50.899239 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:50.899239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:50.899378 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:50.899553 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:50.899553 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:50.899554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:50.899680 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:50.899746 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:50.899841 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 20:12:50.899918 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:12:50.921098 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:38575
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:41425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:51.003993 27385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:51.004186 27385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:51.004220 27385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:51.005378 27385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:51.005420 27385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:51.005447 27385 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 20:12:51.005462 27385 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:12:51.005486 27385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:12:51.006983 27385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:41425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.200.254:38575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:51.007242 27385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:51.007463 27385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:51.012531 27393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:51.012606 27385 server_base.cc:1034] running on GCE node
W20260807 20:12:51.012543 27396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:51.012543 27394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:51.012939 27385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:51.013216 27385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:51.014406 27385 hybrid_clock.cc:648] HybridClock initialized: now 1786133571014387 us; error 29 us; skew 500 ppm
Aug 07 20:12:51 dist-test-slave-2j7r krb5kdc[26956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133571, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:51.017278 27385 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:12:51.017761 27385 webserver.cc:460] Webserver started at http://127.23.200.194:37423/ using document root <none> and password file <none>
I20260807 20:12:51.018005 27385 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:51.018055 27385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:51.018178 27385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:51.019181 27385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "f293453ddb794d6eb0fac83824db851b"
format_stamp: "Formatted at 2026-08-07 20:12:51 on dist-test-slave-2j7r"
server_key: "4387442eeac270f12e7d19b4f5d07cd7"
server_key_iv: "d87d7f09b5ddc45c2e08e4db1bd95e94"
server_key_version: "encryptionkey@0"
I20260807 20:12:51.019510 27385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "f293453ddb794d6eb0fac83824db851b"
format_stamp: "Formatted at 2026-08-07 20:12:51 on dist-test-slave-2j7r"
server_key: "4387442eeac270f12e7d19b4f5d07cd7"
server_key_iv: "d87d7f09b5ddc45c2e08e4db1bd95e94"
server_key_version: "encryptionkey@0"
I20260807 20:12:51.020818 27385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:51.021579 27408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:51.021728 27385 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:51.021818 27385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "f293453ddb794d6eb0fac83824db851b"
format_stamp: "Formatted at 2026-08-07 20:12:51 on dist-test-slave-2j7r"
server_key: "4387442eeac270f12e7d19b4f5d07cd7"
server_key_iv: "d87d7f09b5ddc45c2e08e4db1bd95e94"
server_key_version: "encryptionkey@0"
I20260807 20:12:51.021885 27385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:51.041699 27385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:51.043677 27385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:51.043818 27385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:51.044423 27385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:51.044473 27385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:51.044502 27385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:51.044518 27385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 20:12:51 dist-test-slave-2j7r krb5kdc[26956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133571, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:51.051405 27385 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:42233
I20260807 20:12:51.051506 27608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:42233 every 8 connection(s)
I20260807 20:12:51.051796 27385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:12:51.054119 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.044812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:59543 (local address 127.23.200.254:38575)
0807 20:12:51.044870 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.044872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.045080 (+ 208us) server_negotiation.cc:407] Connection header received
0807 20:12:51.045709 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.045711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.045724 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.045777 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:51.046492 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.047020 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.048003 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.048190 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.048245 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.051386 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:51.051388 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:51.051389 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:51.051397 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:51.052954 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:51.053374 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:51.053375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:51.053375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:51.053438 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:51.053631 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:51.053632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:51.053632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:51.053735 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:51.053778 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.053985 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.054052 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:12:51.054293 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.044793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:38575 (local address 127.23.200.194:59543)
0807 20:12:51.044957 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.044996 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.045604 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.045856 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.045860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.046183 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:51.046405 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.046410 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.047095 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.047096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.047924 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.047926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.048257 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.048258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.048393 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.048916 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:51.048928 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:51.051282 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:51.053032 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:51.053039 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:51.053040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:51.053297 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:51.053488 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:51.053489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:51.053489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:51.053589 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:51.053772 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:51.053776 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:51.053889 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.053968 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.054091 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260807 20:12:51.054795 27609 heartbeater.cc:347] Connected to a master server at 127.23.200.254:38575
I20260807 20:12:51.054880 27609 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:51.055064 27609 heartbeater.cc:511] Master 127.23.200.254:38575 requested a full tablet report, sending...
I20260807 20:12:51.055482 27032 ts_manager.cc:194] Registered new tserver with Master: f293453ddb794d6eb0fac83824db851b (127.23.200.194:42233)
I20260807 20:12:51.055891 27032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:59543
I20260807 20:12:51.056389 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 27385
I20260807 20:12:51.056485 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:12:51.056667 24355 external_mini_cluster.cc:1428] Setting key 69ad6e04c0e85adb0457339edffa56fd
I20260807 20:12:51.061302 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.055852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:58741 (local address 127.23.200.254:38575)
0807 20:12:51.055897 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.055899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.055966 (+ 67us) server_negotiation.cc:407] Connection header received
0807 20:12:51.056031 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.056032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.056043 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.056099 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:51.056419 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.057072 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.057854 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.058108 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.058175 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.059031 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:51.059033 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:51.059033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:51.059040 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:51.060254 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:51.060610 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:51.060612 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:51.060612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:51.060667 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:51.060845 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:51.060846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:51.060846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:51.060976 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:51.061023 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.061185 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.061242 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:12:51.061369 27619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.055774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:38575 (local address 127.23.200.194:58741)
0807 20:12:51.055900 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.055919 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.055981 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.056103 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.056104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.056270 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:51.056340 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.056342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.057221 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.057223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.057774 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.057776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.058174 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.058174 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.058224 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.058621 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:51.058626 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:51.058945 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:51.060331 (+ 1386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:51.060332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:51.060332 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:51.060531 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:51.060724 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:51.060725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:51.060725 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:51.060790 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:51.061029 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:51.061030 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:51.061077 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.061165 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.061261 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:12:51.084082 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:38575
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:41425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:51.166455 27623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:51.166667 27623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:51.166692 27623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:51.167865 27623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:51.167908 27623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:51.167928 27623 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 20:12:51.167938 27623 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:12:51.167954 27623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:12:51.169446 27623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:41425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.200.254:38575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:51.169668 27623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:51.169857 27623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:12:51.174750 27623 server_base.cc:1034] running on GCE node
W20260807 20:12:51.174770 27632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:51.174824 27631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:51.174770 27640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:51.175159 27623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:51.175390 27623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:51.176546 27623 hybrid_clock.cc:648] HybridClock initialized: now 1786133571176534 us; error 29 us; skew 500 ppm
Aug 07 20:12:51 dist-test-slave-2j7r krb5kdc[26956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133571, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:51.179068 27623 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:12:51.179513 27623 webserver.cc:460] Webserver started at http://127.23.200.195:33423/ using document root <none> and password file <none>
I20260807 20:12:51.179730 27623 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:51.179774 27623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:51.179893 27623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:51.180895 27623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "03890ff11d3d467a913266dda3140aa3"
format_stamp: "Formatted at 2026-08-07 20:12:51 on dist-test-slave-2j7r"
server_key: "8549cf6014553fd19d0df5174abe8fe7"
server_key_iv: "e51cc2bce7c0d8b4f29f3c4a0586945c"
server_key_version: "encryptionkey@0"
I20260807 20:12:51.181241 27623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "03890ff11d3d467a913266dda3140aa3"
format_stamp: "Formatted at 2026-08-07 20:12:51 on dist-test-slave-2j7r"
server_key: "8549cf6014553fd19d0df5174abe8fe7"
server_key_iv: "e51cc2bce7c0d8b4f29f3c4a0586945c"
server_key_version: "encryptionkey@0"
I20260807 20:12:51.182515 27623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:51.183221 27649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:51.183369 27623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:51.183465 27623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "03890ff11d3d467a913266dda3140aa3"
format_stamp: "Formatted at 2026-08-07 20:12:51 on dist-test-slave-2j7r"
server_key: "8549cf6014553fd19d0df5174abe8fe7"
server_key_iv: "e51cc2bce7c0d8b4f29f3c4a0586945c"
server_key_version: "encryptionkey@0"
I20260807 20:12:51.183524 27623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:51.194332 27623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:51.196074 27623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:51.196208 27623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:51.196787 27623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:51.196836 27623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:51.196875 27623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:51.196899 27623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 20:12:51 dist-test-slave-2j7r krb5kdc[26956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133571, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:51.204172 27623 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:37101
I20260807 20:12:51.204499 27823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:37101 every 8 connection(s)
I20260807 20:12:51.204541 27623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:12:51.206749 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.197175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:60875 (local address 127.23.200.254:38575)
0807 20:12:51.197235 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.197237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.197520 (+ 283us) server_negotiation.cc:407] Connection header received
0807 20:12:51.198178 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.198179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.198191 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.198233 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:51.199111 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.199743 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.201252 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.201460 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.201524 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.203935 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:51.203937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:51.203937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:51.203945 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:51.205465 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:51.205859 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:51.205861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:51.205861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:51.205920 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:51.206123 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:51.206124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:51.206124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:51.206227 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:51.206273 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.206609 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.206681 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:12:51.206928 27663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.197186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:38575 (local address 127.23.200.195:60875)
0807 20:12:51.197373 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.197423 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.198076 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.198316 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.198320 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.198696 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:51.199000 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.199008 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.199824 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.199825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.201142 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.201145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.201558 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.201560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.201694 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.202309 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:51.202332 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:51.203823 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:51.205555 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:51.205560 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:51.205562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:51.205782 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:51.205988 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:51.205988 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:51.205989 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:51.206083 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:51.206324 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:51.206331 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:51.206465 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.206571 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.206713 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":91,"threads_started":1}
I20260807 20:12:51.207733 27824 heartbeater.cc:347] Connected to a master server at 127.23.200.254:38575
I20260807 20:12:51.207821 27824 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:51.208007 27824 heartbeater.cc:511] Master 127.23.200.254:38575 requested a full tablet report, sending...
I20260807 20:12:51.208422 27032 ts_manager.cc:194] Registered new tserver with Master: 03890ff11d3d467a913266dda3140aa3 (127.23.200.195:37101)
I20260807 20:12:51.208889 27032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:60875
I20260807 20:12:51.209581 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 27623
I20260807 20:12:51.209679 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:12:51.209867 24355 external_mini_cluster.cc:1428] Setting key af63e54a3e7f15fbb727df3d6094a5cd
I20260807 20:12:51.210732 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:12:51.214813 27834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.208766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:38575 (local address 127.23.200.195:50391)
0807 20:12:51.208925 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.208936 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.209017 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.209180 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.209180 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.209378 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:51.209446 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.209449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.210344 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.210345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.211033 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.211036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.211356 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.211357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.211432 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.211826 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:51.211831 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:51.212132 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:51.213846 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:51.213847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:51.213847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:51.214046 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:51.214231 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:51.214232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:51.214232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:51.214287 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:51.214521 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:51.214522 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:51.214570 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.214622 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.214705 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
I20260807 20:12:51.214838 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.208825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:50391 (local address 127.23.200.254:38575)
0807 20:12:51.208864 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.208866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.208989 (+ 123us) server_negotiation.cc:407] Connection header received
0807 20:12:51.209073 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.209075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.209110 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.209172 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:51.209533 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.210194 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.211105 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.211299 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.211353 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.212210 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:51.212212 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:51.212212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:51.212219 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:51.213753 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:51.214111 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:51.214112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:51.214113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:51.214174 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:51.214335 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:51.214336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:51.214336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:51.214466 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:51.214529 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.214659 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.214757 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 07 20:12:51 dist-test-slave-2j7r krb5kdc[26956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 20:12:51 dist-test-slave-2j7r krb5kdc[26956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:51.235948 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.229013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35478 (local address 127.23.200.254:38575)
0807 20:12:51.229074 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.229076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.229121 (+ 45us) server_negotiation.cc:407] Connection header received
0807 20:12:51.229150 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.229151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.229162 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.229217 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:51.229880 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.230389 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.231211 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.231418 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.231477 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.233649 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:51.233651 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:51.233651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:51.233660 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:51.234908 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:51.235227 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:51.235228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:51.235228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:51.235280 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:51.235452 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:51.235453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:51.235453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:51.235562 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:51.235653 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.235761 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.235842 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:12:51.237564 27032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:12:51.239135 27032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:12:51.245399 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.241390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:42233 (local address 127.0.0.1:33706)
0807 20:12:51.241546 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.241583 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.241650 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.242041 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.242045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.242048 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:51.242151 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.242153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.243116 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.243117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.244242 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.244245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.245059 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.245060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.245187 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.245200 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.245263 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.245316 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":2,"thread_start_us":111,"threads_started":1}
I20260807 20:12:51.245424 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.241649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33706 (local address 127.23.200.194:42233)
0807 20:12:51.241819 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.241825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.241835 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:51.241875 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.241876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.241892 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.242003 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:51.242219 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.243044 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.244337 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.244986 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.245049 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.245138 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.245286 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.245324 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260807 20:12:51.245800 27853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.241390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:37101 (local address 127.0.0.1:38042)
0807 20:12:51.241752 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.241762 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.241805 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.242046 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.242046 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.242048 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:51.242214 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.242216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.243577 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.243579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.244622 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.244625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.245545 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.245546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.245629 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.245633 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.245681 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.245719 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":58,"thread_start_us":47,"threads_started":1}
I20260807 20:12:51.245963 27857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.241627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43430 (local address 127.23.200.193:46743)
0807 20:12:51.242147 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.242151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.242161 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:51.242257 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.242260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.242275 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.242340 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:51.242537 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.243597 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.244600 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.245355 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.245431 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.245777 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.245804 (+ 27us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.245842 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":110,"threads_started":1}
I20260807 20:12:51.246094 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.241401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:46743 (local address 127.0.0.1:43430)
0807 20:12:51.242120 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.242130 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.242172 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.242416 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.242417 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.242419 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:51.242468 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.242470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.243669 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.243671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.244526 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.244529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.245544 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.245545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.245629 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.245633 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.245677 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.245719 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260807 20:12:51.246096 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.241572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38042 (local address 127.23.200.195:37101)
0807 20:12:51.241903 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.241907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.241915 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:12:51.241952 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.241953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.241967 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.242037 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:51.242300 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.243477 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.244700 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.245410 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.245466 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.245515 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.245901 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.245944 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":93,"threads_started":1}
I20260807 20:12:51.246344 27497 tablet_service.cc:1467] Processing CreateTablet for tablet 8a627ae7dec24083bfac3ca0370068c7 (DEFAULT_TABLE table=test-table [id=e48ee526aee3460dbddf945b4421d7ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:51.246713 27497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a627ae7dec24083bfac3ca0370068c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:51.246728 27260 tablet_service.cc:1467] Processing CreateTablet for tablet 8a627ae7dec24083bfac3ca0370068c7 (DEFAULT_TABLE table=test-table [id=e48ee526aee3460dbddf945b4421d7ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:51.247073 27260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a627ae7dec24083bfac3ca0370068c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:51.249620 27858 tablet_bootstrap.cc:492] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b: Bootstrap starting.
I20260807 20:12:51.249702 27722 tablet_service.cc:1467] Processing CreateTablet for tablet 8a627ae7dec24083bfac3ca0370068c7 (DEFAULT_TABLE table=test-table [id=e48ee526aee3460dbddf945b4421d7ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:51.250067 27722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a627ae7dec24083bfac3ca0370068c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:51.250272 27858 tablet_bootstrap.cc:654] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:51.250530 27858 log.cc:824] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:51.251189 27858 tablet_bootstrap.cc:492] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b: No bootstrap required, opened a new log
I20260807 20:12:51.251235 27858 ts_tablet_manager.cc:1397] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:12:51.252461 27862 tablet_bootstrap.cc:492] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3: Bootstrap starting.
I20260807 20:12:51.252753 27859 tablet_bootstrap.cc:492] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Bootstrap starting.
I20260807 20:12:51.252662 27858 raft_consensus.cc:348] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } }
I20260807 20:12:51.252823 27858 raft_consensus.cc:374] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:51.252871 27858 raft_consensus.cc:729] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f293453ddb794d6eb0fac83824db851b, State: Initialized, Role: FOLLOWER
I20260807 20:12:51.253008 27858 consensus_queue.cc:260] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } }
I20260807 20:12:51.253298 27858 ts_tablet_manager.cc:1428] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:12:51.253436 27609 heartbeater.cc:503] Master 127.23.200.254:38575 was elected leader, sending a full tablet report...
I20260807 20:12:51.253556 27862 tablet_bootstrap.cc:654] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:51.253556 27859 tablet_bootstrap.cc:654] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:51.253896 27862 log.cc:824] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:51.253907 27859 log.cc:824] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:51.254690 27862 tablet_bootstrap.cc:492] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3: No bootstrap required, opened a new log
I20260807 20:12:51.254699 27859 tablet_bootstrap.cc:492] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b: No bootstrap required, opened a new log
I20260807 20:12:51.254757 27862 ts_tablet_manager.cc:1397] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:12:51.254758 27859 ts_tablet_manager.cc:1397] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:12:51.256707 27859 raft_consensus.cc:348] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } }
I20260807 20:12:51.256678 27862 raft_consensus.cc:348] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } }
I20260807 20:12:51.256836 27859 raft_consensus.cc:374] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:51.256836 27862 raft_consensus.cc:374] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:51.256870 27862 raft_consensus.cc:729] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03890ff11d3d467a913266dda3140aa3, State: Initialized, Role: FOLLOWER
I20260807 20:12:51.256870 27859 raft_consensus.cc:729] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cd3bdfda33543cdb8582c3e4eb33c8b, State: Initialized, Role: FOLLOWER
I20260807 20:12:51.256985 27859 consensus_queue.cc:260] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } }
I20260807 20:12:51.256985 27862 consensus_queue.cc:260] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } }
I20260807 20:12:51.257269 27862 ts_tablet_manager.cc:1428] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:12:51.257269 27859 ts_tablet_manager.cc:1428] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260807 20:12:51.257432 27824 heartbeater.cc:503] Master 127.23.200.254:38575 was elected leader, sending a full tablet report...
I20260807 20:12:51.257467 27371 heartbeater.cc:503] Master 127.23.200.254:38575 was elected leader, sending a full tablet report...
W20260807 20:12:51.302448 27611 tablet.cc:2369] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:12:51.372184 27863 raft_consensus.cc:482] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:12:51.372303 27863 raft_consensus.cc:504] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } }
I20260807 20:12:51.372630 27863 leader_election.cc:288] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03890ff11d3d467a913266dda3140aa3 (127.23.200.195:37101), 8cd3bdfda33543cdb8582c3e4eb33c8b (127.23.200.193:46743)
I20260807 20:12:51.375672 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.372825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:37101 (local address 127.23.200.194:43833)
0807 20:12:51.372906 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.372918 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.372967 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.373191 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.373192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.373193 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:51.373264 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.373265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.373902 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.373903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.374731 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.374734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.375431 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.375432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.375514 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.375519 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.375568 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.375606 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 20:12:51.375689 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.372915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:43833 (local address 127.23.200.195:37101)
0807 20:12:51.372970 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.372973 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.373037 (+ 64us) server_negotiation.cc:407] Connection header received
0807 20:12:51.373073 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.373073 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.373123 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.373178 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:51.373325 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.373829 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.374849 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.375330 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.375389 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.375397 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.375580 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.375620 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 20:12:51.375978 27868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.372874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:46743 (local address 127.23.200.194:60047)
0807 20:12:51.373062 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.373071 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.373137 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.373414 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.373415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.373427 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:51.373495 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.373496 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.374207 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.374208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.375011 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.375014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.375581 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.375582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.375661 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.375665 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.375850 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.375892 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":141,"threads_started":1}
I20260807 20:12:51.376091 27857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.372997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:60047 (local address 127.23.200.193:46743)
0807 20:12:51.373065 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.373067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.373132 (+ 65us) server_negotiation.cc:407] Connection header received
0807 20:12:51.373341 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.373342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.373350 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.373390 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:51.373552 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.374092 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.375111 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.375494 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.375564 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.375573 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.375735 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.376028 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 20:12:51.376040 27758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a627ae7dec24083bfac3ca0370068c7" candidate_uuid: "f293453ddb794d6eb0fac83824db851b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03890ff11d3d467a913266dda3140aa3" is_pre_election: true
I20260807 20:12:51.376201 27758 raft_consensus.cc:2454] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f293453ddb794d6eb0fac83824db851b in term 0.
I20260807 20:12:51.376361 27287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a627ae7dec24083bfac3ca0370068c7" candidate_uuid: "f293453ddb794d6eb0fac83824db851b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" is_pre_election: true
I20260807 20:12:51.376461 27416 leader_election.cc:302] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03890ff11d3d467a913266dda3140aa3, f293453ddb794d6eb0fac83824db851b; no voters:
I20260807 20:12:51.376514 27287 raft_consensus.cc:2454] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f293453ddb794d6eb0fac83824db851b in term 0.
I20260807 20:12:51.376573 27863 raft_consensus.cc:2781] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:12:51.376637 27863 raft_consensus.cc:482] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:12:51.376667 27863 raft_consensus.cc:3037] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:51.377430 27863 raft_consensus.cc:504] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } }
I20260807 20:12:51.377553 27863 leader_election.cc:288] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [CANDIDATE]: Term 1 election: Requested vote from peers 03890ff11d3d467a913266dda3140aa3 (127.23.200.195:37101), 8cd3bdfda33543cdb8582c3e4eb33c8b (127.23.200.193:46743)
I20260807 20:12:51.377723 27758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a627ae7dec24083bfac3ca0370068c7" candidate_uuid: "f293453ddb794d6eb0fac83824db851b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03890ff11d3d467a913266dda3140aa3"
I20260807 20:12:51.377777 27287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a627ae7dec24083bfac3ca0370068c7" candidate_uuid: "f293453ddb794d6eb0fac83824db851b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b"
I20260807 20:12:51.377815 27758 raft_consensus.cc:3037] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:51.377848 27287 raft_consensus.cc:3037] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:51.378705 27758 raft_consensus.cc:2454] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f293453ddb794d6eb0fac83824db851b in term 1.
I20260807 20:12:51.378705 27287 raft_consensus.cc:2454] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f293453ddb794d6eb0fac83824db851b in term 1.
I20260807 20:12:51.379158 27411 leader_election.cc:302] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cd3bdfda33543cdb8582c3e4eb33c8b, f293453ddb794d6eb0fac83824db851b; no voters:
I20260807 20:12:51.379290 27863 raft_consensus.cc:2781] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:51.379447 27863 raft_consensus.cc:686] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [term 1 LEADER]: Becoming Leader. State: Replica: f293453ddb794d6eb0fac83824db851b, State: Running, Role: LEADER
I20260807 20:12:51.379652 27863 consensus_queue.cc:237] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } }
I20260807 20:12:51.380528 27031 catalog_manager.cc:5495] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b reported cstate change: term changed from 0 to 1, leader changed from <none> to f293453ddb794d6eb0fac83824db851b (127.23.200.194). New cstate: current_term: 1 leader_uuid: "f293453ddb794d6eb0fac83824db851b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } health_report { overall_health: UNKNOWN } } }
W20260807 20:12:51.391175 27373 tablet.cc:2369] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 20:12:51 dist-test-slave-2j7r krb5kdc[26956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133570, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:51.432581 27110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.425998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35492 (local address 127.23.200.254:38575)
0807 20:12:51.426090 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.426093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.426202 (+ 109us) server_negotiation.cc:407] Connection header received
0807 20:12:51.426277 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.426278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.426288 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.426339 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:51.426866 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.427362 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.428027 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.428197 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.428250 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.430230 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:51.430231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:51.430232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:51.430239 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:51.431514 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:51.431849 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:51.431850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:51.431850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:51.431903 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:51.432060 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:51.432061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:51.432062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:51.432167 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:51.432222 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.432370 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.432469 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":59}
I20260807 20:12:51.432637 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.425913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:38575 (local address 127.0.0.1:35492)
0807 20:12:51.426125 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.426136 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.426216 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.426389 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.426390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.426727 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:51.426778 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.426780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.427431 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.427432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.427960 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.427961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.428253 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.428254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.428310 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.428789 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:51.428793 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:51.430151 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:51.431588 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:51.431597 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:51.431600 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:51.431787 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:51.431955 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:51.431956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:51.431956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:51.432013 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:51.432217 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:51.432227 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:51.432274 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.432339 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.432523 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":77,"thread_start_us":107,"threads_started":1}
I20260807 20:12:51.434829 27031 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:35492:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 20:12:51.435037 27031 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:12:51.436517 27497 tablet_service.cc:1467] Processing CreateTablet for tablet 3bcacea0d185407a82992a694d2a30d2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3da48801092d42a99ee0411f2bc8a7cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:12:51.436546 27722 tablet_service.cc:1467] Processing CreateTablet for tablet 3bcacea0d185407a82992a694d2a30d2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3da48801092d42a99ee0411f2bc8a7cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:12:51.436582 27260 tablet_service.cc:1467] Processing CreateTablet for tablet 3bcacea0d185407a82992a694d2a30d2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3da48801092d42a99ee0411f2bc8a7cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:12:51.436700 27497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bcacea0d185407a82992a694d2a30d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:51.436710 27722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bcacea0d185407a82992a694d2a30d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:51.436733 27260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bcacea0d185407a82992a694d2a30d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:51.437994 27858 tablet_bootstrap.cc:492] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b: Bootstrap starting.
I20260807 20:12:51.438184 27862 tablet_bootstrap.cc:492] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3: Bootstrap starting.
I20260807 20:12:51.438439 27858 tablet_bootstrap.cc:654] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:51.438468 27859 tablet_bootstrap.cc:492] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Bootstrap starting.
I20260807 20:12:51.438715 27862 tablet_bootstrap.cc:654] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:51.438766 27859 tablet_bootstrap.cc:654] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:51.439162 27858 tablet_bootstrap.cc:492] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b: No bootstrap required, opened a new log
I20260807 20:12:51.439211 27858 ts_tablet_manager.cc:1397] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:51.439333 27859 tablet_bootstrap.cc:492] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b: No bootstrap required, opened a new log
I20260807 20:12:51.439366 27859 ts_tablet_manager.cc:1397] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260807 20:12:51.439339 27858 raft_consensus.cc:348] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.439396 27858 raft_consensus.cc:374] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:51.439411 27858 raft_consensus.cc:729] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f293453ddb794d6eb0fac83824db851b, State: Initialized, Role: FOLLOWER
I20260807 20:12:51.439448 27858 consensus_queue.cc:260] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.439478 27859 raft_consensus.cc:348] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.439528 27858 ts_tablet_manager.cc:1428] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:51.439533 27859 raft_consensus.cc:374] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:51.439551 27859 raft_consensus.cc:729] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cd3bdfda33543cdb8582c3e4eb33c8b, State: Initialized, Role: FOLLOWER
I20260807 20:12:51.439603 27859 consensus_queue.cc:260] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.439620 27863 tablet_replica.cc:439] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } }
I20260807 20:12:51.439697 27863 tablet_replica.cc:442] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:12:51.439710 27859 ts_tablet_manager.cc:1428] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:51.439836 27867 tablet_replica.cc:439] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } }
I20260807 20:12:51.439893 27867 tablet_replica.cc:442] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:12:51.440326 27862 tablet_bootstrap.cc:492] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3: No bootstrap required, opened a new log
I20260807 20:12:51.440376 27862 ts_tablet_manager.cc:1397] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260807 20:12:51.440548 27862 raft_consensus.cc:348] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.440605 27862 raft_consensus.cc:374] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:51.440618 27862 raft_consensus.cc:729] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03890ff11d3d467a913266dda3140aa3, State: Initialized, Role: FOLLOWER
I20260807 20:12:51.440646 27862 consensus_queue.cc:260] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.440722 27862 ts_tablet_manager.cc:1428] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:51.440804 27866 tablet_replica.cc:439] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } }
I20260807 20:12:51.440877 27866 tablet_replica.cc:442] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3: This TxnStatusTablet replica's current role is: FOLLOWER
W20260807 20:12:51.455793 27825 tablet.cc:2369] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:12:51.565151 27866 raft_consensus.cc:482] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:12:51.565263 27866 raft_consensus.cc:504] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.565670 27866 leader_election.cc:288] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cd3bdfda33543cdb8582c3e4eb33c8b (127.23.200.193:46743), f293453ddb794d6eb0fac83824db851b (127.23.200.194:42233)
I20260807 20:12:51.569552 27857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.566025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:51373 (local address 127.23.200.193:46743)
0807 20:12:51.566084 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.566085 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.566192 (+ 107us) server_negotiation.cc:407] Connection header received
0807 20:12:51.566224 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.566225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.566233 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.566282 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:51.566408 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.567100 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.568581 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.569168 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.569277 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.569286 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.569455 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.569495 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:12:51.569540 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.565987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:60465 (local address 127.23.200.194:42233)
0807 20:12:51.566033 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.566035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.566047 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:12:51.566131 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.566133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.566141 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.566201 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:51.566355 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.567120 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.568581 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.569168 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.569247 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.569256 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.569423 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.569469 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:12:51.569540 27663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.565871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:42233 (local address 127.23.200.195:60465)
0807 20:12:51.565971 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.565984 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.566045 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.566214 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.566215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.566217 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:51.566288 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.566290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.567187 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.567187 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.568487 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.568491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.569245 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.569247 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.569346 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.569351 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.569411 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.569454 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":58}
I20260807 20:12:51.569551 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.565930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:46743 (local address 127.23.200.195:51373)
0807 20:12:51.566125 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.566133 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.566182 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.566290 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.566291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.566292 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:51.566347 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.566348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.567172 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.567172 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.568487 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.568490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.569302 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.569303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.569358 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.569360 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.569441 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.569470 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":74,"thread_start_us":123,"threads_started":1}
I20260807 20:12:51.569850 27287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bcacea0d185407a82992a694d2a30d2" candidate_uuid: "03890ff11d3d467a913266dda3140aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" is_pre_election: true
I20260807 20:12:51.569960 27287 raft_consensus.cc:2454] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03890ff11d3d467a913266dda3140aa3 in term 0.
I20260807 20:12:51.570014 27527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bcacea0d185407a82992a694d2a30d2" candidate_uuid: "03890ff11d3d467a913266dda3140aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f293453ddb794d6eb0fac83824db851b" is_pre_election: true
I20260807 20:12:51.570122 27527 raft_consensus.cc:2454] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03890ff11d3d467a913266dda3140aa3 in term 0.
I20260807 20:12:51.570140 27655 leader_election.cc:302] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03890ff11d3d467a913266dda3140aa3, 8cd3bdfda33543cdb8582c3e4eb33c8b; no voters:
I20260807 20:12:51.570281 27866 raft_consensus.cc:2781] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:12:51.570323 27866 raft_consensus.cc:482] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:12:51.570350 27866 raft_consensus.cc:3037] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:51.571027 27866 raft_consensus.cc:504] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.571146 27866 leader_election.cc:288] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [CANDIDATE]: Term 1 election: Requested vote from peers 8cd3bdfda33543cdb8582c3e4eb33c8b (127.23.200.193:46743), f293453ddb794d6eb0fac83824db851b (127.23.200.194:42233)
I20260807 20:12:51.571314 27527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bcacea0d185407a82992a694d2a30d2" candidate_uuid: "03890ff11d3d467a913266dda3140aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f293453ddb794d6eb0fac83824db851b"
I20260807 20:12:51.571308 27287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bcacea0d185407a82992a694d2a30d2" candidate_uuid: "03890ff11d3d467a913266dda3140aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b"
I20260807 20:12:51.571391 27527 raft_consensus.cc:3037] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:51.571391 27287 raft_consensus.cc:3037] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:51.572160 27287 raft_consensus.cc:2454] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03890ff11d3d467a913266dda3140aa3 in term 1.
I20260807 20:12:51.572160 27527 raft_consensus.cc:2454] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03890ff11d3d467a913266dda3140aa3 in term 1.
I20260807 20:12:51.572320 27655 leader_election.cc:302] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03890ff11d3d467a913266dda3140aa3, 8cd3bdfda33543cdb8582c3e4eb33c8b; no voters:
I20260807 20:12:51.572443 27866 raft_consensus.cc:2781] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:51.572594 27866 raft_consensus.cc:686] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [term 1 LEADER]: Becoming Leader. State: Replica: 03890ff11d3d467a913266dda3140aa3, State: Running, Role: LEADER
I20260807 20:12:51.572695 27866 consensus_queue.cc:237] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.573276 27889 tablet_replica.cc:439] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3: TxnStatusTablet state changed. Reason: New leader 03890ff11d3d467a913266dda3140aa3. Latest consensus state: current_term: 1 leader_uuid: "03890ff11d3d467a913266dda3140aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } }
I20260807 20:12:51.573360 27889 tablet_replica.cc:442] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:12:51.573588 27891 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 20:12:51.573550 27031 catalog_manager.cc:5495] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03890ff11d3d467a913266dda3140aa3 (127.23.200.195). New cstate: current_term: 1 leader_uuid: "03890ff11d3d467a913266dda3140aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } health_report { overall_health: HEALTHY } } }
I20260807 20:12:51.573704 27891 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 20:12:51.586905 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.584015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:37101 (local address 127.0.0.1:38058)
0807 20:12:51.584093 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.584105 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.584149 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.584301 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.584302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.584303 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:12:51.584358 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.584360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.585256 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.585257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.585822 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.585824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.586325 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.586326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.586398 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.586415 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:12:51.586727 (+ 312us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:12:51.586740 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.586785 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.586831 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 20:12:51.586916 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.584076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38058 (local address 127.23.200.195:37101)
0807 20:12:51.584124 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.584126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.584138 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:12:51.584211 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.584212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.584232 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.584301 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:51.584406 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.585189 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.585888 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.586234 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.586294 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.586496 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:51.586645 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:51.586726 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.586791 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.586815 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:12:51.588758 27287 raft_consensus.cc:1261] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 FOLLOWER]: Refusing update from remote peer 03890ff11d3d467a913266dda3140aa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:51.588754 27527 raft_consensus.cc:1261] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b [term 1 FOLLOWER]: Refusing update from remote peer 03890ff11d3d467a913266dda3140aa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:51.589056 27889 consensus_queue.cc:1035] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:51.589283 27889 consensus_queue.cc:1035] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:51.590288 27863 tablet_replica.cc:439] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b: TxnStatusTablet state changed. Reason: New leader 03890ff11d3d467a913266dda3140aa3. Latest consensus state: current_term: 1 leader_uuid: "03890ff11d3d467a913266dda3140aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } }
I20260807 20:12:51.590384 27863 tablet_replica.cc:442] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:12:51.590545 27867 tablet_replica.cc:439] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b: TxnStatusTablet state changed. Reason: New leader 03890ff11d3d467a913266dda3140aa3. Latest consensus state: current_term: 1 leader_uuid: "03890ff11d3d467a913266dda3140aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } }
I20260807 20:12:51.590636 27867 tablet_replica.cc:442] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:12:51.591348 27866 tablet_replica.cc:439] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "03890ff11d3d467a913266dda3140aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } }
I20260807 20:12:51.591442 27866 tablet_replica.cc:442] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:12:51.591512 27863 tablet_replica.cc:439] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "03890ff11d3d467a913266dda3140aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } }
I20260807 20:12:51.591579 27863 tablet_replica.cc:442] T 3bcacea0d185407a82992a694d2a30d2 P f293453ddb794d6eb0fac83824db851b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:12:51.591710 27896 mvcc.cc:205] Tried to move back new op lower bound from 7316003109224808448 to 7316003109162352640. Current Snapshot: MvccSnapshot[applied={T|T < 7316003109224808448}]
I20260807 20:12:51.591980 27898 mvcc.cc:205] Tried to move back new op lower bound from 7316003109224808448 to 7316003109162352640. Current Snapshot: MvccSnapshot[applied={T|T < 7316003109224808448}]
I20260807 20:12:51.592564 27889 tablet_replica.cc:439] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "03890ff11d3d467a913266dda3140aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } }
I20260807 20:12:51.592644 27889 tablet_replica.cc:442] T 3bcacea0d185407a82992a694d2a30d2 P 03890ff11d3d467a913266dda3140aa3: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:12:51.592943 27867 tablet_replica.cc:439] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "03890ff11d3d467a913266dda3140aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } } }
I20260807 20:12:51.593029 27867 tablet_replica.cc:442] T 3bcacea0d185407a82992a694d2a30d2 P 8cd3bdfda33543cdb8582c3e4eb33c8b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:12:51.594311 27901 mvcc.cc:205] Tried to move back new op lower bound from 7316003109224808448 to 7316003109162352640. Current Snapshot: MvccSnapshot[applied={T|T < 7316003109224808448}]
I20260807 20:12:51.596504 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.594100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33712 (local address 127.23.200.194:42233)
0807 20:12:51.594153 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.594155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.594162 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:12:51.594191 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.594192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.594210 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.594283 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:51.594469 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.595007 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.595823 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.596026 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.596080 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.596218 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:51.596317 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:51.596381 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.596420 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.596440 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 20:12:51.601045 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.598970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:60207 (local address 127.23.200.195:37101)
0807 20:12:51.599028 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.599030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.599038 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:12:51.599079 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.599080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.599089 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.599134 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:51.599247 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.599685 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.600434 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.600610 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.600663 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.600831 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:51.600860 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:51.600899 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.600955 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.600976 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:12:51.601071 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.598795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:37101 (local address 127.23.200.194:60207)
0807 20:12:51.598966 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.598976 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.599028 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.599134 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.599135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.599136 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:12:51.599191 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.599192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.599750 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.599751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.600331 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.600333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.600673 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.600674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.600739 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.600753 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:12:51.600913 (+ 160us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:12:51.600915 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.600954 (+ 39us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.600997 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260807 20:12:51.603394 27287 raft_consensus.cc:1261] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 FOLLOWER]: Refusing update from remote peer f293453ddb794d6eb0fac83824db851b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:51.603480 27758 raft_consensus.cc:1261] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Refusing update from remote peer f293453ddb794d6eb0fac83824db851b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:51.603641 27863 consensus_queue.cc:1035] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:51.603767 27869 consensus_queue.cc:1035] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [LEADER]: Connected to new peer: Peer: permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:51.612512 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.610181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:33453 (local address 127.23.200.194:42233)
0807 20:12:51.610241 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.610243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.610250 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:12:51.610267 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.610268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.610277 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.610318 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:51.610476 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.610914 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.611942 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.612119 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.612170 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.612338 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:51.612363 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:51.612395 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.612443 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.612460 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 20:12:51.612552 27834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.610006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:42233 (local address 127.23.200.195:33453)
0807 20:12:51.610063 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.610074 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.610116 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.610353 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.610354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.610355 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:12:51.610415 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.610417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.610972 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.610973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.611855 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.611858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.612181 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.612182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.612247 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.612264 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:12:51.612398 (+ 134us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:12:51.612400 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.612437 (+ 37us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.612476 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260807 20:12:51.621956 27032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:51.622074 27032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:51.622814 27032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5: Sending DeleteTablet for 3 replicas of tablet 8a627ae7dec24083bfac3ca0370068c7
I20260807 20:12:51.623139 27260 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a627ae7dec24083bfac3ca0370068c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:51 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:43430
I20260807 20:12:51.623175 27722 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a627ae7dec24083bfac3ca0370068c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:51 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:38042
I20260807 20:12:51.623387 27497 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a627ae7dec24083bfac3ca0370068c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:51 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:33706
I20260807 20:12:51.623411 27032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:12:51.623545 27032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:12:51.623647 27940 tablet_replica.cc:330] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3: stopping tablet replica
I20260807 20:12:51.623842 27940 raft_consensus.cc:2229] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:51.623971 27940 raft_consensus.cc:2258] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:51.624459 27940 ts_tablet_manager.cc:1905] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:51.624698 27260 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1ceeeb4cea4c06b5cc12266bcb1ec3 (DEFAULT_TABLE table=test-table [id=3b638972c0294105b4acb38c87cee4f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:51.624698 27497 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1ceeeb4cea4c06b5cc12266bcb1ec3 (DEFAULT_TABLE table=test-table [id=3b638972c0294105b4acb38c87cee4f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:51.624828 27497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1ceeeb4cea4c06b5cc12266bcb1ec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:51.624830 27260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1ceeeb4cea4c06b5cc12266bcb1ec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:51.626206 27858 tablet_bootstrap.cc:492] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b: Bootstrap starting.
I20260807 20:12:51.626206 27859 tablet_bootstrap.cc:492] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Bootstrap starting.
I20260807 20:12:51.626616 27859 tablet_bootstrap.cc:654] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:51.626634 27858 tablet_bootstrap.cc:654] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:51.626614 27940 ts_tablet_manager.cc:1918] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:12:51.626808 27940 log.cc:1196] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/8a627ae7dec24083bfac3ca0370068c7
I20260807 20:12:51.627132 27940 ts_tablet_manager.cc:1939] T 8a627ae7dec24083bfac3ca0370068c7 P 03890ff11d3d467a913266dda3140aa3: Deleting consensus metadata
I20260807 20:12:51.627246 27722 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1ceeeb4cea4c06b5cc12266bcb1ec3 (DEFAULT_TABLE table=test-table [id=3b638972c0294105b4acb38c87cee4f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:51.627374 27722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1ceeeb4cea4c06b5cc12266bcb1ec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:51.627712 27013 catalog_manager.cc:4841] TS 03890ff11d3d467a913266dda3140aa3 (127.23.200.195:37101): tablet 8a627ae7dec24083bfac3ca0370068c7 (table test-table [id=e48ee526aee3460dbddf945b4421d7ac]) successfully deleted
I20260807 20:12:51.628191 27862 tablet_bootstrap.cc:492] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3: Bootstrap starting.
I20260807 20:12:51.628605 27862 tablet_bootstrap.cc:654] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:51.629359 27862 tablet_bootstrap.cc:492] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3: No bootstrap required, opened a new log
I20260807 20:12:51.629415 27862 ts_tablet_manager.cc:1397] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260807 20:12:51.629554 27862 raft_consensus.cc:348] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.629616 27862 raft_consensus.cc:374] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:51.629634 27862 raft_consensus.cc:729] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03890ff11d3d467a913266dda3140aa3, State: Initialized, Role: FOLLOWER
I20260807 20:12:51.629673 27862 consensus_queue.cc:260] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.629762 27862 ts_tablet_manager.cc:1428] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:51.633208 27941 tablet_replica.cc:330] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b: stopping tablet replica
I20260807 20:12:51.633337 27941 raft_consensus.cc:2229] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:12:51.633458 27941 raft_consensus.cc:2258] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:51.633519 27858 tablet_bootstrap.cc:492] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b: No bootstrap required, opened a new log
I20260807 20:12:51.633564 27858 ts_tablet_manager.cc:1397] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260807 20:12:51.633708 27858 raft_consensus.cc:348] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.633785 27858 raft_consensus.cc:374] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:51.633811 27858 raft_consensus.cc:729] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f293453ddb794d6eb0fac83824db851b, State: Initialized, Role: FOLLOWER
I20260807 20:12:51.633853 27858 consensus_queue.cc:260] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.633909 27859 tablet_bootstrap.cc:492] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b: No bootstrap required, opened a new log
I20260807 20:12:51.633946 27858 ts_tablet_manager.cc:1428] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:51.633955 27859 ts_tablet_manager.cc:1397] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.000s
I20260807 20:12:51.634115 27859 raft_consensus.cc:348] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.634194 27859 raft_consensus.cc:374] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:51.634220 27859 raft_consensus.cc:729] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cd3bdfda33543cdb8582c3e4eb33c8b, State: Initialized, Role: FOLLOWER
I20260807 20:12:51.634261 27859 consensus_queue.cc:260] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.634268 27941 ts_tablet_manager.cc:1905] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:51.634354 27859 ts_tablet_manager.cc:1428] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:51.636289 27941 ts_tablet_manager.cc:1918] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:12:51.636363 27941 log.cc:1196] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/8a627ae7dec24083bfac3ca0370068c7
I20260807 20:12:51.636620 27941 ts_tablet_manager.cc:1939] T 8a627ae7dec24083bfac3ca0370068c7 P f293453ddb794d6eb0fac83824db851b: Deleting consensus metadata
I20260807 20:12:51.637038 27008 catalog_manager.cc:4841] TS f293453ddb794d6eb0fac83824db851b (127.23.200.194:42233): tablet 8a627ae7dec24083bfac3ca0370068c7 (table test-table [id=e48ee526aee3460dbddf945b4421d7ac]) successfully deleted
I20260807 20:12:51.637238 27934 tablet_replica.cc:330] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b: stopping tablet replica
I20260807 20:12:51.637398 27934 raft_consensus.cc:2229] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:51.637499 27934 raft_consensus.cc:2258] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:51.637959 27934 ts_tablet_manager.cc:1905] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:51.639635 27934 ts_tablet_manager.cc:1918] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:12:51.639758 27934 log.cc:1196] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/wal/wals/8a627ae7dec24083bfac3ca0370068c7
I20260807 20:12:51.640070 27934 ts_tablet_manager.cc:1939] T 8a627ae7dec24083bfac3ca0370068c7 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Deleting consensus metadata
I20260807 20:12:51.640529 27010 catalog_manager.cc:4841] TS 8cd3bdfda33543cdb8582c3e4eb33c8b (127.23.200.193:46743): tablet 8a627ae7dec24083bfac3ca0370068c7 (table test-table [id=e48ee526aee3460dbddf945b4421d7ac]) successfully deleted
I20260807 20:12:51.880163 27867 raft_consensus.cc:482] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:12:51.880283 27867 raft_consensus.cc:504] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.880653 27867 leader_election.cc:288] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f293453ddb794d6eb0fac83824db851b (127.23.200.194:42233), 03890ff11d3d467a913266dda3140aa3 (127.23.200.195:37101)
I20260807 20:12:51.884774 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.880922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:39579 (local address 127.23.200.195:37101)
0807 20:12:51.880983 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.880986 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.881113 (+ 127us) server_negotiation.cc:407] Connection header received
0807 20:12:51.881179 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.881180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.881190 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.881254 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:51.881482 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.882206 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.883756 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.884348 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.884432 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.884441 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.884656 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.884695 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:12:51.884774 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.880955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:38781 (local address 127.23.200.194:42233)
0807 20:12:51.881010 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.881012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.881107 (+ 95us) server_negotiation.cc:407] Connection header received
0807 20:12:51.881179 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.881180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.881192 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.881254 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:51.881471 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.882266 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.883756 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.884355 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.884430 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.884441 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.884652 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.884693 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:12:51.884806 27950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.880854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:42233 (local address 127.23.200.193:38781)
0807 20:12:51.881013 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.881025 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.881116 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.881274 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.881275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.881278 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:51.881394 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.881396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.882332 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.882333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.883652 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.883656 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.884456 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.884458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.884566 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.884568 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.884636 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.884687 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":48,"thread_start_us":81,"threads_started":1}
I20260807 20:12:51.884815 27949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.880853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:37101 (local address 127.23.200.193:39579)
0807 20:12:51.881013 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.881025 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.881116 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.881274 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.881275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.881279 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:51.881417 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.881418 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.882285 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.882287 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.883652 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.883656 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.884457 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.884458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.884563 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.884570 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.884636 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.884687 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":80,"thread_start_us":71,"threads_started":1}
I20260807 20:12:51.885170 27527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1ceeeb4cea4c06b5cc12266bcb1ec3" candidate_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f293453ddb794d6eb0fac83824db851b" is_pre_election: true
I20260807 20:12:51.885334 27527 raft_consensus.cc:2454] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cd3bdfda33543cdb8582c3e4eb33c8b in term 0.
I20260807 20:12:51.885300 27758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1ceeeb4cea4c06b5cc12266bcb1ec3" candidate_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03890ff11d3d467a913266dda3140aa3" is_pre_election: true
I20260807 20:12:51.885391 27758 raft_consensus.cc:2454] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cd3bdfda33543cdb8582c3e4eb33c8b in term 0.
I20260807 20:12:51.885552 27176 leader_election.cc:302] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cd3bdfda33543cdb8582c3e4eb33c8b, f293453ddb794d6eb0fac83824db851b; no voters:
I20260807 20:12:51.885699 27867 raft_consensus.cc:2781] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:12:51.885751 27867 raft_consensus.cc:482] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:12:51.885764 27867 raft_consensus.cc:3037] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:51.886510 27867 raft_consensus.cc:504] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.886665 27867 leader_election.cc:288] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [CANDIDATE]: Term 1 election: Requested vote from peers f293453ddb794d6eb0fac83824db851b (127.23.200.194:42233), 03890ff11d3d467a913266dda3140aa3 (127.23.200.195:37101)
I20260807 20:12:51.886806 27527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1ceeeb4cea4c06b5cc12266bcb1ec3" candidate_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f293453ddb794d6eb0fac83824db851b"
I20260807 20:12:51.886827 27758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1ceeeb4cea4c06b5cc12266bcb1ec3" candidate_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03890ff11d3d467a913266dda3140aa3"
I20260807 20:12:51.886902 27527 raft_consensus.cc:3037] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:51.886904 27758 raft_consensus.cc:3037] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:51.887534 27758 raft_consensus.cc:2454] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cd3bdfda33543cdb8582c3e4eb33c8b in term 1.
I20260807 20:12:51.887580 27527 raft_consensus.cc:2454] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cd3bdfda33543cdb8582c3e4eb33c8b in term 1.
I20260807 20:12:51.887693 27181 leader_election.cc:302] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03890ff11d3d467a913266dda3140aa3, 8cd3bdfda33543cdb8582c3e4eb33c8b; no voters:
I20260807 20:12:51.887800 27867 raft_consensus.cc:2781] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:51.887964 27867 raft_consensus.cc:686] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 LEADER]: Becoming Leader. State: Replica: 8cd3bdfda33543cdb8582c3e4eb33c8b, State: Running, Role: LEADER
I20260807 20:12:51.888031 27867 consensus_queue.cc:237] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:51.888783 27031 catalog_manager.cc:5495] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8cd3bdfda33543cdb8582c3e4eb33c8b (127.23.200.193). New cstate: current_term: 1 leader_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } health_report { overall_health: UNKNOWN } } }
I20260807 20:12:51.914587 27857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.912245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43442 (local address 127.23.200.193:46743)
0807 20:12:51.912290 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.912292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.912299 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:12:51.912368 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.912369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.912391 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.912463 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:51.912554 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.913130 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.913909 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.914086 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.914128 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.914255 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:51.914363 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:51.914414 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.914467 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.914486 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:12:51.919289 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.917178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:39753 (local address 127.23.200.195:37101)
0807 20:12:51.917222 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.917223 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.917346 (+ 123us) server_negotiation.cc:407] Connection header received
0807 20:12:51.917400 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.917401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.917416 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.917454 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:51.917547 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.918008 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.918741 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.918927 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.918969 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.919117 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:51.919140 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:51.919171 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.919213 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.919228 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:12:51.919312 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.917116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:37101 (local address 127.23.200.193:39753)
0807 20:12:51.917284 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.917295 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.917347 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.917457 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.917458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.917459 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:12:51.917510 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.917512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.918075 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.918076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.918680 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.918682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.918999 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.919000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.919045 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.919059 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:12:51.919182 (+ 123us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:12:51.919183 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.919211 (+ 28us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.919244 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260807 20:12:51.921512 27758 raft_consensus.cc:1261] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Refusing update from remote peer 8cd3bdfda33543cdb8582c3e4eb33c8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:51.921512 27527 raft_consensus.cc:1261] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b [term 1 FOLLOWER]: Refusing update from remote peer 8cd3bdfda33543cdb8582c3e4eb33c8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:51.921795 27952 consensus_queue.cc:1035] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:51.921932 27867 consensus_queue.cc:1035] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:51.929193 27857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.926867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:55955 (local address 127.23.200.193:46743)
0807 20:12:51.926915 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:51.926917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:51.926925 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:12:51.926989 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:51.926990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:51.927000 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:51.927043 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:51.927221 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.927769 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.928457 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.928707 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.928775 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.928947 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:51.928982 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:51.929028 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:51.929077 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 20:12:51.929128 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:12:51.929220 27834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:51.926828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:46743 (local address 127.23.200.195:55955)
0807 20:12:51.926880 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:51.926899 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:51.926937 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:51.927099 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:51.927100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:51.927101 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:12:51.927149 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.927150 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.927828 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.927828 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.928371 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:51.928373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:51.928774 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:51.928775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:51.928833 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:51.928857 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:12:51.929029 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:12:51.929031 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:12:51.929073 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 20:12:51.929145 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 20:12:51.929508 27901 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 20:12:53.449759 27032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:53.449879 27032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:53.450704 27032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5: Sending DeleteTablet for 3 replicas of tablet 0a1ceeeb4cea4c06b5cc12266bcb1ec3
I20260807 20:12:53.450994 27260 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1ceeeb4cea4c06b5cc12266bcb1ec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:53 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:43430
I20260807 20:12:53.451004 27722 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1ceeeb4cea4c06b5cc12266bcb1ec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:53 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:38042
I20260807 20:12:53.451277 27497 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1ceeeb4cea4c06b5cc12266bcb1ec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:53 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:33706
I20260807 20:12:53.451386 28001 tablet_replica.cc:330] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3: stopping tablet replica
I20260807 20:12:53.451447 28001 raft_consensus.cc:2229] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:53.451489 28001 raft_consensus.cc:2258] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:53.451529 28002 tablet_replica.cc:330] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b: stopping tablet replica
I20260807 20:12:53.451587 28002 raft_consensus.cc:2229] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:53.451627 28002 raft_consensus.cc:2258] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:53.451771 28001 ts_tablet_manager.cc:1905] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:53.451874 28002 ts_tablet_manager.cc:1905] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:53.453439 28000 tablet_replica.cc:330] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b: stopping tablet replica
I20260807 20:12:53.453507 28000 raft_consensus.cc:2229] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:12:53.453660 28002 ts_tablet_manager.cc:1918] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 20:12:53.453660 28001 ts_tablet_manager.cc:1918] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 20:12:53.453615 28000 raft_consensus.cc:2258] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:53.453752 28001 log.cc:1196] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/0a1ceeeb4cea4c06b5cc12266bcb1ec3
I20260807 20:12:53.453763 28002 log.cc:1196] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/0a1ceeeb4cea4c06b5cc12266bcb1ec3
I20260807 20:12:53.453994 28000 ts_tablet_manager.cc:1905] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:53.454051 28002 ts_tablet_manager.cc:1939] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P f293453ddb794d6eb0fac83824db851b: Deleting consensus metadata
I20260807 20:12:53.454088 28001 ts_tablet_manager.cc:1939] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 03890ff11d3d467a913266dda3140aa3: Deleting consensus metadata
I20260807 20:12:53.454418 27013 catalog_manager.cc:4841] TS 03890ff11d3d467a913266dda3140aa3 (127.23.200.195:37101): tablet 0a1ceeeb4cea4c06b5cc12266bcb1ec3 (table test-table [id=3b638972c0294105b4acb38c87cee4f3]) successfully deleted
I20260807 20:12:53.454525 27008 catalog_manager.cc:4841] TS f293453ddb794d6eb0fac83824db851b (127.23.200.194:42233): tablet 0a1ceeeb4cea4c06b5cc12266bcb1ec3 (table test-table [id=3b638972c0294105b4acb38c87cee4f3]) successfully deleted
I20260807 20:12:53.455643 28000 ts_tablet_manager.cc:1918] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 20:12:53.455693 28000 log.cc:1196] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/wal/wals/0a1ceeeb4cea4c06b5cc12266bcb1ec3
I20260807 20:12:53.455907 28000 ts_tablet_manager.cc:1939] T 0a1ceeeb4cea4c06b5cc12266bcb1ec3 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Deleting consensus metadata
I20260807 20:12:53.456331 27010 catalog_manager.cc:4841] TS 8cd3bdfda33543cdb8582c3e4eb33c8b (127.23.200.193:46743): tablet 0a1ceeeb4cea4c06b5cc12266bcb1ec3 (table test-table [id=3b638972c0294105b4acb38c87cee4f3]) successfully deleted
Aug 07 20:12:53 dist-test-slave-2j7r krb5kdc[26956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133573, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 20:12:53 dist-test-slave-2j7r krb5kdc[26956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133573, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:53.477696 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:53.470528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35508 (local address 127.23.200.254:38575)
0807 20:12:53.470673 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:53.470675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:53.470686 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:12:53.470711 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:53.470713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:53.470724 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:53.470802 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:53.471401 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:53.471913 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:53.472657 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:53.472882 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:53.472936 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:53.475120 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:53.475121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:53.475122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:53.475129 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:53.476643 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:53.476964 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:53.476965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:53.476966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:53.477028 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:53.477228 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:53.477228 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:53.477229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:53.477395 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:53.477474 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:53.477558 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 20:12:53.477624 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260807 20:12:53.479159 27032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:12:53.479316 27032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:12:53.480722 27260 tablet_service.cc:1467] Processing CreateTablet for tablet 87ecb7790b4b481e882675260ac82995 (DEFAULT_TABLE table=test-table [id=6091d9dc4ea647d2827261a92f290695]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:53.480739 27497 tablet_service.cc:1467] Processing CreateTablet for tablet 87ecb7790b4b481e882675260ac82995 (DEFAULT_TABLE table=test-table [id=6091d9dc4ea647d2827261a92f290695]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:53.480754 27722 tablet_service.cc:1467] Processing CreateTablet for tablet 87ecb7790b4b481e882675260ac82995 (DEFAULT_TABLE table=test-table [id=6091d9dc4ea647d2827261a92f290695]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:53.480873 27260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87ecb7790b4b481e882675260ac82995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:53.480875 27497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87ecb7790b4b481e882675260ac82995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:53.480883 27722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87ecb7790b4b481e882675260ac82995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:53.481972 28018 tablet_bootstrap.cc:492] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3: Bootstrap starting.
I20260807 20:12:53.482357 28020 tablet_bootstrap.cc:492] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Bootstrap starting.
I20260807 20:12:53.482506 28018 tablet_bootstrap.cc:654] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:53.482699 28020 tablet_bootstrap.cc:654] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:53.482992 28021 tablet_bootstrap.cc:492] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b: Bootstrap starting.
I20260807 20:12:53.483100 28018 tablet_bootstrap.cc:492] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3: No bootstrap required, opened a new log
I20260807 20:12:53.483150 28018 ts_tablet_manager.cc:1397] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:53.483350 28021 tablet_bootstrap.cc:654] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:53.483310 28018 raft_consensus.cc:348] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.483390 28018 raft_consensus.cc:374] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:53.483410 28018 raft_consensus.cc:729] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03890ff11d3d467a913266dda3140aa3, State: Initialized, Role: FOLLOWER
I20260807 20:12:53.483484 28018 consensus_queue.cc:260] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.483596 28018 ts_tablet_manager.cc:1428] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:53.484054 28020 tablet_bootstrap.cc:492] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b: No bootstrap required, opened a new log
I20260807 20:12:53.484082 28021 tablet_bootstrap.cc:492] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b: No bootstrap required, opened a new log
I20260807 20:12:53.484129 28021 ts_tablet_manager.cc:1397] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:53.484140 28020 ts_tablet_manager.cc:1397] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:12:53.484251 28021 raft_consensus.cc:348] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.484329 28021 raft_consensus.cc:374] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:53.484349 28021 raft_consensus.cc:729] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f293453ddb794d6eb0fac83824db851b, State: Initialized, Role: FOLLOWER
I20260807 20:12:53.484302 28020 raft_consensus.cc:348] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.484398 28021 consensus_queue.cc:260] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.484441 28020 raft_consensus.cc:374] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:53.484469 28020 raft_consensus.cc:729] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cd3bdfda33543cdb8582c3e4eb33c8b, State: Initialized, Role: FOLLOWER
I20260807 20:12:53.484527 28020 consensus_queue.cc:260] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.484575 28021 ts_tablet_manager.cc:1428] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:53.484624 28020 ts_tablet_manager.cc:1428] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:53.637068 28029 raft_consensus.cc:482] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:12:53.637197 28029 raft_consensus.cc:504] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.637362 28029 leader_election.cc:288] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cd3bdfda33543cdb8582c3e4eb33c8b (127.23.200.193:46743), 03890ff11d3d467a913266dda3140aa3 (127.23.200.195:37101)
I20260807 20:12:53.637665 27758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ecb7790b4b481e882675260ac82995" candidate_uuid: "f293453ddb794d6eb0fac83824db851b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03890ff11d3d467a913266dda3140aa3" is_pre_election: true
I20260807 20:12:53.637663 27287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ecb7790b4b481e882675260ac82995" candidate_uuid: "f293453ddb794d6eb0fac83824db851b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" is_pre_election: true
I20260807 20:12:53.637759 27758 raft_consensus.cc:2454] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f293453ddb794d6eb0fac83824db851b in term 0.
I20260807 20:12:53.637761 27287 raft_consensus.cc:2454] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f293453ddb794d6eb0fac83824db851b in term 0.
I20260807 20:12:53.637933 27416 leader_election.cc:302] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03890ff11d3d467a913266dda3140aa3, f293453ddb794d6eb0fac83824db851b; no voters:
I20260807 20:12:53.638039 28029 raft_consensus.cc:2781] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:12:53.638080 28029 raft_consensus.cc:482] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:12:53.638093 28029 raft_consensus.cc:3037] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:53.638782 28029 raft_consensus.cc:504] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.638922 28029 leader_election.cc:288] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [CANDIDATE]: Term 1 election: Requested vote from peers 8cd3bdfda33543cdb8582c3e4eb33c8b (127.23.200.193:46743), 03890ff11d3d467a913266dda3140aa3 (127.23.200.195:37101)
I20260807 20:12:53.639048 27287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ecb7790b4b481e882675260ac82995" candidate_uuid: "f293453ddb794d6eb0fac83824db851b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b"
I20260807 20:12:53.639099 27758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ecb7790b4b481e882675260ac82995" candidate_uuid: "f293453ddb794d6eb0fac83824db851b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03890ff11d3d467a913266dda3140aa3"
I20260807 20:12:53.639135 27287 raft_consensus.cc:3037] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:53.639164 27758 raft_consensus.cc:3037] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:53.639919 27758 raft_consensus.cc:2454] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f293453ddb794d6eb0fac83824db851b in term 1.
I20260807 20:12:53.639920 27287 raft_consensus.cc:2454] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f293453ddb794d6eb0fac83824db851b in term 1.
I20260807 20:12:53.640100 27416 leader_election.cc:302] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03890ff11d3d467a913266dda3140aa3, f293453ddb794d6eb0fac83824db851b; no voters:
I20260807 20:12:53.640192 28029 raft_consensus.cc:2781] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:53.640338 28029 raft_consensus.cc:686] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [term 1 LEADER]: Becoming Leader. State: Replica: f293453ddb794d6eb0fac83824db851b, State: Running, Role: LEADER
I20260807 20:12:53.640383 28029 consensus_queue.cc:237] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.641016 27031 catalog_manager.cc:5495] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b reported cstate change: term changed from 0 to 1, leader changed from <none> to f293453ddb794d6eb0fac83824db851b (127.23.200.194). New cstate: current_term: 1 leader_uuid: "f293453ddb794d6eb0fac83824db851b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } health_report { overall_health: UNKNOWN } } }
I20260807 20:12:53.667929 28034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:53.665798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33714 (local address 127.23.200.194:42233)
0807 20:12:53.665980 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:53.665982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:53.665989 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:12:53.666009 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:53.666010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:53.666034 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:53.666093 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:53.666233 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:53.666706 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:53.667391 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:53.667584 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:53.667634 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:53.667723 (+ 89us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:53.667748 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:53.667777 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:53.667808 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 20:12:53.667825 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
I20260807 20:12:53.670259 27287 raft_consensus.cc:1261] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 FOLLOWER]: Refusing update from remote peer f293453ddb794d6eb0fac83824db851b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:53.670282 27758 raft_consensus.cc:1261] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Refusing update from remote peer f293453ddb794d6eb0fac83824db851b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:53.670470 28031 consensus_queue.cc:1035] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:53.670579 28029 consensus_queue.cc:1035] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [LEADER]: Connected to new peer: Peer: permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:53.680922 27031 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:53.681020 27031 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:53.681653 27031 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5: Sending DeleteTablet for 3 replicas of tablet 87ecb7790b4b481e882675260ac82995
I20260807 20:12:53.681876 27260 tablet_service.cc:1514] Processing DeleteTablet for tablet 87ecb7790b4b481e882675260ac82995 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:53 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:43430
I20260807 20:12:53.681926 27497 tablet_service.cc:1514] Processing DeleteTablet for tablet 87ecb7790b4b481e882675260ac82995 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:53 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:33706
I20260807 20:12:53.682031 28002 tablet_replica.cc:330] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b: stopping tablet replica
I20260807 20:12:53.682029 28000 tablet_replica.cc:330] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b: stopping tablet replica
I20260807 20:12:53.682030 27722 tablet_service.cc:1514] Processing DeleteTablet for tablet 87ecb7790b4b481e882675260ac82995 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:53 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:38042
I20260807 20:12:53.682089 28002 raft_consensus.cc:2229] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:12:53.682089 28000 raft_consensus.cc:2229] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:53.682127 28000 raft_consensus.cc:2258] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:53.682129 28001 tablet_replica.cc:330] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3: stopping tablet replica
I20260807 20:12:53.682149 28002 raft_consensus.cc:2258] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:53.682186 28001 raft_consensus.cc:2229] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:53.682226 28001 raft_consensus.cc:2258] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:53.682384 27031 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 20:12:53.682512 28000 ts_tablet_manager.cc:1905] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:53.682539 28002 ts_tablet_manager.cc:1905] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 20:12:53.682576 27031 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:12:53.683640 27497 tablet_service.cc:1467] Processing CreateTablet for tablet a8ffd4a0688e42e2b55816fd5e9e0a12 (DEFAULT_TABLE table=test-table [id=f25a474f057c42b79c304dfbec84a969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:53.683899 27260 tablet_service.cc:1467] Processing CreateTablet for tablet a8ffd4a0688e42e2b55816fd5e9e0a12 (DEFAULT_TABLE table=test-table [id=f25a474f057c42b79c304dfbec84a969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:53.684005 27260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8ffd4a0688e42e2b55816fd5e9e0a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:53.683998 27722 tablet_service.cc:1467] Processing CreateTablet for tablet a8ffd4a0688e42e2b55816fd5e9e0a12 (DEFAULT_TABLE table=test-table [id=f25a474f057c42b79c304dfbec84a969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:53.684113 27722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8ffd4a0688e42e2b55816fd5e9e0a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:53.684183 28000 ts_tablet_manager.cc:1918] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:12:53.684234 28000 log.cc:1196] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-0/wal/wals/87ecb7790b4b481e882675260ac82995
I20260807 20:12:53.684423 28000 ts_tablet_manager.cc:1939] T 87ecb7790b4b481e882675260ac82995 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Deleting consensus metadata
I20260807 20:12:53.684775 27010 catalog_manager.cc:4841] TS 8cd3bdfda33543cdb8582c3e4eb33c8b (127.23.200.193:46743): tablet 87ecb7790b4b481e882675260ac82995 (table test-table [id=6091d9dc4ea647d2827261a92f290695]) successfully deleted
I20260807 20:12:53.685209 28020 tablet_bootstrap.cc:492] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Bootstrap starting.
I20260807 20:12:53.685293 28002 ts_tablet_manager.cc:1918] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:12:53.685300 28018 tablet_bootstrap.cc:492] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3: Bootstrap starting.
I20260807 20:12:53.685339 28002 log.cc:1196] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/87ecb7790b4b481e882675260ac82995
I20260807 20:12:53.685529 28020 tablet_bootstrap.cc:654] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:53.685560 28002 ts_tablet_manager.cc:1939] T 87ecb7790b4b481e882675260ac82995 P f293453ddb794d6eb0fac83824db851b: Deleting consensus metadata
I20260807 20:12:53.685591 28018 tablet_bootstrap.cc:654] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:53.685931 27497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8ffd4a0688e42e2b55816fd5e9e0a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:53.686167 28020 tablet_bootstrap.cc:492] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 8cd3bdfda33543cdb8582c3e4eb33c8b: No bootstrap required, opened a new log
I20260807 20:12:53.686205 28020 ts_tablet_manager.cc:1397] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:53.686247 28018 tablet_bootstrap.cc:492] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3: No bootstrap required, opened a new log
I20260807 20:12:53.686290 28018 ts_tablet_manager.cc:1397] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:53.686290 28020 raft_consensus.cc:348] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.686337 28020 raft_consensus.cc:374] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:53.686348 28020 raft_consensus.cc:729] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cd3bdfda33543cdb8582c3e4eb33c8b, State: Initialized, Role: FOLLOWER
I20260807 20:12:53.686384 28020 consensus_queue.cc:260] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 8cd3bdfda33543cdb8582c3e4eb33c8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.686401 28018 raft_consensus.cc:348] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.686461 28018 raft_consensus.cc:374] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:53.686477 28018 raft_consensus.cc:729] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03890ff11d3d467a913266dda3140aa3, State: Initialized, Role: FOLLOWER
I20260807 20:12:53.686471 28020 ts_tablet_manager.cc:1428] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 8cd3bdfda33543cdb8582c3e4eb33c8b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:53.686524 28018 consensus_queue.cc:260] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.686614 28018 ts_tablet_manager.cc:1428] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:53.687124 28021 tablet_bootstrap.cc:492] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b: Bootstrap starting.
I20260807 20:12:53.687288 27008 catalog_manager.cc:4841] TS f293453ddb794d6eb0fac83824db851b (127.23.200.194:42233): tablet 87ecb7790b4b481e882675260ac82995 (table test-table [id=6091d9dc4ea647d2827261a92f290695]) successfully deleted
I20260807 20:12:53.687444 28021 tablet_bootstrap.cc:654] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:53.687919 28001 ts_tablet_manager.cc:1905] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:53.688118 28021 tablet_bootstrap.cc:492] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b: No bootstrap required, opened a new log
I20260807 20:12:53.688161 28021 ts_tablet_manager.cc:1397] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:53.688261 28021 raft_consensus.cc:348] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.688316 28021 raft_consensus.cc:374] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:53.688328 28021 raft_consensus.cc:729] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f293453ddb794d6eb0fac83824db851b, State: Initialized, Role: FOLLOWER
I20260807 20:12:53.688360 28021 consensus_queue.cc:260] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.688445 28021 ts_tablet_manager.cc:1428] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:53.689477 28001 ts_tablet_manager.cc:1918] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:12:53.689527 28001 log.cc:1196] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/87ecb7790b4b481e882675260ac82995
I20260807 20:12:53.689702 28001 ts_tablet_manager.cc:1939] T 87ecb7790b4b481e882675260ac82995 P 03890ff11d3d467a913266dda3140aa3: Deleting consensus metadata
I20260807 20:12:53.689985 27013 catalog_manager.cc:4841] TS 03890ff11d3d467a913266dda3140aa3 (127.23.200.195:37101): tablet 87ecb7790b4b481e882675260ac82995 (table test-table [id=6091d9dc4ea647d2827261a92f290695]) successfully deleted
I20260807 20:12:53.736913 28031 raft_consensus.cc:482] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:12:53.737021 28031 raft_consensus.cc:504] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.737221 28031 leader_election.cc:288] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cd3bdfda33543cdb8582c3e4eb33c8b (127.23.200.193:46743), 03890ff11d3d467a913266dda3140aa3 (127.23.200.195:37101)
I20260807 20:12:53.737442 27287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8ffd4a0688e42e2b55816fd5e9e0a12" candidate_uuid: "f293453ddb794d6eb0fac83824db851b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" is_pre_election: true
I20260807 20:12:53.737474 27758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8ffd4a0688e42e2b55816fd5e9e0a12" candidate_uuid: "f293453ddb794d6eb0fac83824db851b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03890ff11d3d467a913266dda3140aa3" is_pre_election: true
I20260807 20:12:53.737545 27287 raft_consensus.cc:2454] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f293453ddb794d6eb0fac83824db851b in term 0.
I20260807 20:12:53.737573 27758 raft_consensus.cc:2454] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f293453ddb794d6eb0fac83824db851b in term 0.
I20260807 20:12:53.737712 27411 leader_election.cc:302] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cd3bdfda33543cdb8582c3e4eb33c8b, f293453ddb794d6eb0fac83824db851b; no voters:
I20260807 20:12:53.737826 28031 raft_consensus.cc:2781] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:12:53.737865 28031 raft_consensus.cc:482] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:12:53.737879 28031 raft_consensus.cc:3037] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:53.738559 28031 raft_consensus.cc:504] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.738673 28031 leader_election.cc:288] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [CANDIDATE]: Term 1 election: Requested vote from peers 8cd3bdfda33543cdb8582c3e4eb33c8b (127.23.200.193:46743), 03890ff11d3d467a913266dda3140aa3 (127.23.200.195:37101)
I20260807 20:12:53.738845 27758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8ffd4a0688e42e2b55816fd5e9e0a12" candidate_uuid: "f293453ddb794d6eb0fac83824db851b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03890ff11d3d467a913266dda3140aa3"
I20260807 20:12:53.738924 27758 raft_consensus.cc:3037] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:53.738867 27287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8ffd4a0688e42e2b55816fd5e9e0a12" candidate_uuid: "f293453ddb794d6eb0fac83824db851b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b"
I20260807 20:12:53.738955 27287 raft_consensus.cc:3037] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:53.739506 27758 raft_consensus.cc:2454] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f293453ddb794d6eb0fac83824db851b in term 1.
I20260807 20:12:53.739565 27287 raft_consensus.cc:2454] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f293453ddb794d6eb0fac83824db851b in term 1.
I20260807 20:12:53.739673 27416 leader_election.cc:302] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03890ff11d3d467a913266dda3140aa3, f293453ddb794d6eb0fac83824db851b; no voters:
I20260807 20:12:53.739799 28031 raft_consensus.cc:2781] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:53.739858 28031 raft_consensus.cc:686] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [term 1 LEADER]: Becoming Leader. State: Replica: f293453ddb794d6eb0fac83824db851b, State: Running, Role: LEADER
I20260807 20:12:53.739923 28031 consensus_queue.cc:237] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } }
I20260807 20:12:53.740483 27031 catalog_manager.cc:5495] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b reported cstate change: term changed from 0 to 1, leader changed from <none> to f293453ddb794d6eb0fac83824db851b (127.23.200.194). New cstate: current_term: 1 leader_uuid: "f293453ddb794d6eb0fac83824db851b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f293453ddb794d6eb0fac83824db851b" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 42233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 } health_report { overall_health: UNKNOWN } } }
I20260807 20:12:53.750321 27758 raft_consensus.cc:1261] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Refusing update from remote peer f293453ddb794d6eb0fac83824db851b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:53.750381 27287 raft_consensus.cc:1261] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 8cd3bdfda33543cdb8582c3e4eb33c8b [term 1 FOLLOWER]: Refusing update from remote peer f293453ddb794d6eb0fac83824db851b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:53.750520 28031 consensus_queue.cc:1035] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [LEADER]: Connected to new peer: Peer: permanent_uuid: "03890ff11d3d467a913266dda3140aa3" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:53.750633 28029 consensus_queue.cc:1035] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cd3bdfda33543cdb8582c3e4eb33c8b" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 46743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 20:12:53.754293 27972 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 20:12:55.270596 27031 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:55.270694 27031 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:55.271492 27031 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfd1e741b37c47e3923ca33f6e04a5d5: Sending DeleteTablet for 3 replicas of tablet a8ffd4a0688e42e2b55816fd5e9e0a12
I20260807 20:12:55.271742 27260 tablet_service.cc:1514] Processing DeleteTablet for tablet a8ffd4a0688e42e2b55816fd5e9e0a12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:55 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:43430
I20260807 20:12:55.271793 27722 tablet_service.cc:1514] Processing DeleteTablet for tablet a8ffd4a0688e42e2b55816fd5e9e0a12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:55 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:38042
I20260807 20:12:55.271934 27497 tablet_service.cc:1514] Processing DeleteTablet for tablet a8ffd4a0688e42e2b55816fd5e9e0a12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:55 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:33706
I20260807 20:12:55.271950 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 27143
I20260807 20:12:55.279953 28090 tablet_replica.cc:330] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b: stopping tablet replica
I20260807 20:12:55.280037 28090 raft_consensus.cc:2229] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:12:55.280128 28090 raft_consensus.cc:2258] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:55.280251 28089 tablet_replica.cc:330] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3: stopping tablet replica
I20260807 20:12:55.280332 28089 raft_consensus.cc:2229] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:55.280426 28089 raft_consensus.cc:2258] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:55.280556 28090 ts_tablet_manager.cc:1905] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:55.280740 28089 ts_tablet_manager.cc:1905] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:55.282773 28090 ts_tablet_manager.cc:1918] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 20:12:55.282773 28089 ts_tablet_manager.cc:1918] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 20:12:55.282845 28089 log.cc:1196] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/a8ffd4a0688e42e2b55816fd5e9e0a12
I20260807 20:12:55.282845 28090 log.cc:1196] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/a8ffd4a0688e42e2b55816fd5e9e0a12
I20260807 20:12:55.283064 28089 ts_tablet_manager.cc:1939] T a8ffd4a0688e42e2b55816fd5e9e0a12 P 03890ff11d3d467a913266dda3140aa3: Deleting consensus metadata
I20260807 20:12:55.283078 28090 ts_tablet_manager.cc:1939] T a8ffd4a0688e42e2b55816fd5e9e0a12 P f293453ddb794d6eb0fac83824db851b: Deleting consensus metadata
I20260807 20:12:55.283401 27013 catalog_manager.cc:4841] TS 03890ff11d3d467a913266dda3140aa3 (127.23.200.195:37101): tablet a8ffd4a0688e42e2b55816fd5e9e0a12 (table test-table [id=f25a474f057c42b79c304dfbec84a969]) successfully deleted
I20260807 20:12:55.283411 27008 catalog_manager.cc:4841] TS f293453ddb794d6eb0fac83824db851b (127.23.200.194:42233): tablet a8ffd4a0688e42e2b55816fd5e9e0a12 (table test-table [id=f25a474f057c42b79c304dfbec84a969]) successfully deleted
W20260807 20:12:55.287472 27010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:46743 (error 108)
I20260807 20:12:55.287627 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 27385
I20260807 20:12:55.290308 28093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:55.290060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:46743 (local address 127.0.0.1:43456)
0807 20:12:55.290201 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:55.290259 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:46743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
W20260807 20:12:55.290591 27010 catalog_manager.cc:4584] TS 8cd3bdfda33543cdb8582c3e4eb33c8b (127.23.200.193:46743): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a8ffd4a0688e42e2b55816fd5e9e0a12: Network error: Client connection negotiation failed: client connection to 127.23.200.193:46743: connect: Connection refused (error 111)
I20260807 20:12:55.297395 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 27623
I20260807 20:12:55.312538 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 26980
2026-08-07T20:12:55Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6013 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:55 dist-test-slave-2j7r krb5kdc[28096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:55 dist-test-slave-2j7r krb5kdc[28096](info): set up 2 sockets
Aug 07 20:12:55 dist-test-slave-2j7r krb5kdc[28096](info): commencing operation
krb5kdc: starting...
W20260807 20:12:56.487694 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:12:56 dist-test-slave-2j7r krb5kdc[28096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:12:56.608809 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:44503
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:40617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:44503
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:56.692185 28121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:56.692480 28121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:56.692503 28121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:56.693758 28121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:56.693809 28121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:56.693822 28121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:56.693833 28121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:56.693845 28121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:56.695379 28121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:40617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:44503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:44503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:56.695632 28121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:56.695855 28121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:56.701186 28129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:56.701221 28121 server_base.cc:1034] running on GCE node
W20260807 20:12:56.701159 28134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:56.701165 28130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:56.701604 28121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:56.701900 28121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:56.703070 28121 hybrid_clock.cc:648] HybridClock initialized: now 1786133576703054 us; error 30 us; skew 500 ppm
Aug 07 20:12:56 dist-test-slave-2j7r krb5kdc[28096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133576, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:56.705729 28121 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:12:56.706225 28121 webserver.cc:460] Webserver started at http://127.23.200.254:42819/ using document root <none> and password file <none>
I20260807 20:12:56.706472 28121 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:56.706523 28121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:56.706650 28121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:56.707700 28121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "62d7b196258e4836b5174c964616a9a0"
format_stamp: "Formatted at 2026-08-07 20:12:56 on dist-test-slave-2j7r"
server_key: "b5fc607ebf4a0454a7839da6e839c63e"
server_key_iv: "3507f878138b63dd2e1b1a36a233664f"
server_key_version: "encryptionkey@0"
I20260807 20:12:56.708042 28121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "62d7b196258e4836b5174c964616a9a0"
format_stamp: "Formatted at 2026-08-07 20:12:56 on dist-test-slave-2j7r"
server_key: "b5fc607ebf4a0454a7839da6e839c63e"
server_key_iv: "3507f878138b63dd2e1b1a36a233664f"
server_key_version: "encryptionkey@0"
I20260807 20:12:56.709345 28121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 20:12:56.710105 28148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:56.710278 28121 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:56.710366 28121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "62d7b196258e4836b5174c964616a9a0"
format_stamp: "Formatted at 2026-08-07 20:12:56 on dist-test-slave-2j7r"
server_key: "b5fc607ebf4a0454a7839da6e839c63e"
server_key_iv: "3507f878138b63dd2e1b1a36a233664f"
server_key_version: "encryptionkey@0"
I20260807 20:12:56.710433 28121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:56.724604 28121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:56.726836 28121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:56.726984 28121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:56.730959 28121 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:44503
I20260807 20:12:56.730996 28217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:44503 every 8 connection(s)
I20260807 20:12:56.731343 28121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:56.732220 28219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:56.734141 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 28121
I20260807 20:12:56.734229 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:12:56.734390 24355 external_mini_cluster.cc:1428] Setting key 9fd64a5495602e7e8da9b78cc213ec14
I20260807 20:12:56.734886 28219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0: Bootstrap starting.
I20260807 20:12:56.735517 28219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:56.735790 28219 log.cc:824] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:56.736590 28219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0: No bootstrap required, opened a new log
I20260807 20:12:56.738792 28219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d7b196258e4836b5174c964616a9a0" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44503 } }
I20260807 20:12:56.738946 28219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:56.738970 28219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d7b196258e4836b5174c964616a9a0, State: Initialized, Role: FOLLOWER
I20260807 20:12:56.739091 28219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d7b196258e4836b5174c964616a9a0" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44503 } }
I20260807 20:12:56.739159 28219 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:56.739183 28219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:56.739228 28219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:56.739913 28219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d7b196258e4836b5174c964616a9a0" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44503 } }
I20260807 20:12:56.740027 28219 leader_election.cc:302] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62d7b196258e4836b5174c964616a9a0; no voters:
I20260807 20:12:56.740203 28219 leader_election.cc:288] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:56.740238 28225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:56.740427 28225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [term 1 LEADER]: Becoming Leader. State: Replica: 62d7b196258e4836b5174c964616a9a0, State: Running, Role: LEADER
I20260807 20:12:56.740482 28219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:56.740595 28225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d7b196258e4836b5174c964616a9a0" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44503 } }
I20260807 20:12:56.740952 28225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62d7b196258e4836b5174c964616a9a0. Latest consensus state: current_term: 1 leader_uuid: "62d7b196258e4836b5174c964616a9a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d7b196258e4836b5174c964616a9a0" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44503 } } }
I20260807 20:12:56.741047 28225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [sys.catalog]: This master's current role is: LEADER
Aug 07 20:12:56 dist-test-slave-2j7r krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:56.741942 28228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62d7b196258e4836b5174c964616a9a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62d7b196258e4836b5174c964616a9a0" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44503 } } }
I20260807 20:12:56.742238 28228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0 [sys.catalog]: This master's current role is: LEADER
W20260807 20:12:56.742321 28263 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:12:56.742395 28263 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:12:56.742452 28237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:56.742647 28237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:56.744122 28237 catalog_manager.cc:1347] Generated new cluster ID: 5e5d6a9dc7044db5835c84dbdd9cbe33
I20260807 20:12:56.744174 28237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:56.745733 28222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:56.734946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39602 (local address 127.23.200.254:44503)
0807 20:12:56.735145 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:56.735155 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:56.735185 (+ 30us) server_negotiation.cc:407] Connection header received
0807 20:12:56.735634 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:56.735644 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:56.735710 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:56.735833 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:56.736567 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:56.737602 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:56.738662 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:56.739143 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:56.739252 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:56.742093 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:56.742097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:56.742098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:56.742111 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:56.744039 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:56.744439 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:56.744443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:56.744443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:56.744553 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:56.744746 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:56.744747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:56.744747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:56.745059 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:56.745261 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:56.745355 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 20:12:56.745511 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260807 20:12:56.751487 28237 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:56.751996 28237 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:56.756245 28237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62d7b196258e4836b5174c964616a9a0: Generated new TSK 0
I20260807 20:12:56.756428 28237 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:12:56.787936 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:44503
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:40617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:56.876310 28268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:56.876572 28268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:56.876609 28268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:56.878211 28268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:56.878270 28268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:56.878310 28268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:12:56.880125 28268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:40617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:44503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:56.880405 28268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:56.880626 28268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:56.885701 28274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:56.885684 28275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:56.885675 28282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:56.885869 28268 server_base.cc:1034] running on GCE node
I20260807 20:12:56.886138 28268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:56.886368 28268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:56.887530 28268 hybrid_clock.cc:648] HybridClock initialized: now 1786133576887516 us; error 28 us; skew 500 ppm
Aug 07 20:12:56 dist-test-slave-2j7r krb5kdc[28096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133576, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:56.890115 28268 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:12:56.890564 28268 webserver.cc:460] Webserver started at http://127.23.200.193:41231/ using document root <none> and password file <none>
I20260807 20:12:56.890799 28268 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:56.890848 28268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:56.890974 28268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:56.892040 28268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "efc7ac0fb4bb402bbdb5b73ba179f23b"
format_stamp: "Formatted at 2026-08-07 20:12:56 on dist-test-slave-2j7r"
server_key: "511927413539b67e41b534d503f18e68"
server_key_iv: "b359ce308b6e7718d11c1e3ce02efd12"
server_key_version: "encryptionkey@0"
I20260807 20:12:56.892360 28268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "efc7ac0fb4bb402bbdb5b73ba179f23b"
format_stamp: "Formatted at 2026-08-07 20:12:56 on dist-test-slave-2j7r"
server_key: "511927413539b67e41b534d503f18e68"
server_key_iv: "b359ce308b6e7718d11c1e3ce02efd12"
server_key_version: "encryptionkey@0"
I20260807 20:12:56.893714 28268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:56.894385 28295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:56.894537 28268 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:56.894621 28268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "efc7ac0fb4bb402bbdb5b73ba179f23b"
format_stamp: "Formatted at 2026-08-07 20:12:56 on dist-test-slave-2j7r"
server_key: "511927413539b67e41b534d503f18e68"
server_key_iv: "b359ce308b6e7718d11c1e3ce02efd12"
server_key_version: "encryptionkey@0"
I20260807 20:12:56.894682 28268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:56.916124 28268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:56.918416 28268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:56.918569 28268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:56.918861 28268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:56.919167 28268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:56.919200 28268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:56.919230 28268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:56.919246 28268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:56.925102 28268 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:40899
I20260807 20:12:56.925173 28476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:40899 every 8 connection(s)
I20260807 20:12:56.925523 28268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
Aug 07 20:12:56 dist-test-slave-2j7r krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133576, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:56.934321 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 28268
I20260807 20:12:56.934415 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:12:56.934576 24355 external_mini_cluster.cc:1428] Setting key 7b330d6b1f139c546b9f1eff29dba442
I20260807 20:12:56.934895 28222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:56.926316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:51049 (local address 127.23.200.254:44503)
0807 20:12:56.926377 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:56.926379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:56.926687 (+ 308us) server_negotiation.cc:407] Connection header received
0807 20:12:56.927171 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:56.927172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:56.927183 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:56.927238 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:56.928211 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:56.928730 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:56.929792 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:56.930010 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:56.930068 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:56.932176 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:56.932178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:56.932178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:56.932185 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:56.933497 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:56.933876 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:56.933877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:56.933877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:56.933937 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:56.934180 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:56.934181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:56.934182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:56.934350 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:56.934426 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:56.934711 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 20:12:56.934803 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 20:12:56.935076 28481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:56.926311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:44503 (local address 127.23.200.193:51049)
0807 20:12:56.926542 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:56.926583 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:56.927089 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:56.927294 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:56.927298 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:56.927870 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:56.928101 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:56.928106 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:56.928820 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:56.928821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:56.929703 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:56.929706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:56.930079 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:56.930080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:56.930177 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:56.930711 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:56.930721 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:56.932084 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:56.933575 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:56.933583 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:56.933585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:56.933799 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:56.933994 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:56.933995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:56.933995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:56.934096 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:56.934442 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:56.934449 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:56.934585 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 20:12:56.934680 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 20:12:56.934853 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":114,"threads_started":1}
I20260807 20:12:56.935554 28479 heartbeater.cc:347] Connected to a master server at 127.23.200.254:44503
I20260807 20:12:56.935657 28479 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:56.935843 28479 heartbeater.cc:511] Master 127.23.200.254:44503 requested a full tablet report, sending...
I20260807 20:12:56.936398 28170 ts_manager.cc:194] Registered new tserver with Master: efc7ac0fb4bb402bbdb5b73ba179f23b (127.23.200.193:40899)
I20260807 20:12:56.937213 28170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:51049
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:12:56.961953 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:44503
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:40617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:57.049353 28485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:57.049561 28485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:57.049592 28485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:57.050746 28485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:57.050788 28485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:57.050822 28485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:12:57.052282 28485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:40617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:44503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:57.052539 28485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:57.052754 28485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:57.057729 28496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:57.057874 28485 server_base.cc:1034] running on GCE node
W20260807 20:12:57.057745 28497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:57.057736 28499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:57.058225 28485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:57.058462 28485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:57.059609 28485 hybrid_clock.cc:648] HybridClock initialized: now 1786133577059596 us; error 29 us; skew 500 ppm
Aug 07 20:12:57 dist-test-slave-2j7r krb5kdc[28096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133577, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:57.062283 28485 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:12:57.062831 28485 webserver.cc:460] Webserver started at http://127.23.200.194:37645/ using document root <none> and password file <none>
I20260807 20:12:57.063056 28485 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:57.063107 28485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:57.063237 28485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:57.064399 28485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "4af0000559d04a588d5de33c46439b8a"
format_stamp: "Formatted at 2026-08-07 20:12:57 on dist-test-slave-2j7r"
server_key: "aca93b9ecfe3d2071c6a1d3bad3bf1b8"
server_key_iv: "fb0c9838e8019eff0e2ffb386e78bf5f"
server_key_version: "encryptionkey@0"
I20260807 20:12:57.064738 28485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "4af0000559d04a588d5de33c46439b8a"
format_stamp: "Formatted at 2026-08-07 20:12:57 on dist-test-slave-2j7r"
server_key: "aca93b9ecfe3d2071c6a1d3bad3bf1b8"
server_key_iv: "fb0c9838e8019eff0e2ffb386e78bf5f"
server_key_version: "encryptionkey@0"
I20260807 20:12:57.066150 28485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:12:57.066962 28508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:57.067129 28485 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:57.067227 28485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "4af0000559d04a588d5de33c46439b8a"
format_stamp: "Formatted at 2026-08-07 20:12:57 on dist-test-slave-2j7r"
server_key: "aca93b9ecfe3d2071c6a1d3bad3bf1b8"
server_key_iv: "fb0c9838e8019eff0e2ffb386e78bf5f"
server_key_version: "encryptionkey@0"
I20260807 20:12:57.067287 28485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:57.098659 28485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:57.100899 28485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:57.101045 28485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:57.101303 28485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:57.101611 28485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:57.101645 28485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:57.101675 28485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:57.101697 28485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:57.107394 28485 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:43207
I20260807 20:12:57.107446 28689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:43207 every 8 connection(s)
I20260807 20:12:57.107757 28485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
Aug 07 20:12:57 dist-test-slave-2j7r krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133577, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:57.116629 28222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:57.108489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:43663 (local address 127.23.200.254:44503)
0807 20:12:57.108555 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:57.108558 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:57.108727 (+ 169us) server_negotiation.cc:407] Connection header received
0807 20:12:57.109269 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:57.109271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:57.109281 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:57.109329 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:57.110221 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:57.110734 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:57.111824 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:57.112029 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:57.112093 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:57.114319 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:57.114321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:57.114321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:57.114329 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:57.115523 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:57.115871 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:57.115872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:57.115872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:57.115917 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:57.116097 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:57.116098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:57.116098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:57.116196 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:57.116239 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:57.116487 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 20:12:57.116560 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:12:57.116799 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:57.108460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:44503 (local address 127.23.200.194:43663)
0807 20:12:57.108613 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:57.108651 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:57.109185 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:57.109397 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:57.109401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:57.109949 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:57.110147 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:57.110152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:57.110807 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:57.110808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:57.111721 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:57.111724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:57.112114 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:57.112115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:57.112230 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:57.112781 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:57.112790 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:57.114232 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:57.115596 (+ 1364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:57.115602 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:57.115604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:57.115815 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:57.115964 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:57.115964 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:57.115964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:57.116056 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:57.116236 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:57.116238 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:57.116351 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 20:12:57.116462 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 20:12:57.116588 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260807 20:12:57.117316 28690 heartbeater.cc:347] Connected to a master server at 127.23.200.254:44503
I20260807 20:12:57.117414 28690 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:57.117587 28690 heartbeater.cc:511] Master 127.23.200.254:44503 requested a full tablet report, sending...
I20260807 20:12:57.117841 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 28485
I20260807 20:12:57.117964 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:12:57.117997 28170 ts_manager.cc:194] Registered new tserver with Master: 4af0000559d04a588d5de33c46439b8a (127.23.200.194:43207)
I20260807 20:12:57.118170 24355 external_mini_cluster.cc:1428] Setting key 868311b4e5c9f82d364037118711db92
I20260807 20:12:57.118346 28170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:43663
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:12:57.143697 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:44503
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:40617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:57.225857 28698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:57.226089 28698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:57.226115 28698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:57.227308 28698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:57.227351 28698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:57.227377 28698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:12:57.228924 28698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:40617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:44503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:57.229198 28698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:57.229440 28698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:57.234611 28706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:57.234614 28713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:57.234668 28698 server_base.cc:1034] running on GCE node
W20260807 20:12:57.234611 28709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:57.235075 28698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:57.235380 28698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:57.236563 28698 hybrid_clock.cc:648] HybridClock initialized: now 1786133577236549 us; error 28 us; skew 500 ppm
Aug 07 20:12:57 dist-test-slave-2j7r krb5kdc[28096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133577, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:57.239341 28698 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:12:57.239777 28698 webserver.cc:460] Webserver started at http://127.23.200.195:36619/ using document root <none> and password file <none>
I20260807 20:12:57.240003 28698 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:57.240048 28698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:57.240168 28698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:57.241292 28698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "7350151b5e834ac29b1bb35e605daead"
format_stamp: "Formatted at 2026-08-07 20:12:57 on dist-test-slave-2j7r"
server_key: "7b63c27fd8826d68fd0071f5a7fb697b"
server_key_iv: "393c6bd40a575e9332896b7614119e77"
server_key_version: "encryptionkey@0"
I20260807 20:12:57.241647 28698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "7350151b5e834ac29b1bb35e605daead"
format_stamp: "Formatted at 2026-08-07 20:12:57 on dist-test-slave-2j7r"
server_key: "7b63c27fd8826d68fd0071f5a7fb697b"
server_key_iv: "393c6bd40a575e9332896b7614119e77"
server_key_version: "encryptionkey@0"
I20260807 20:12:57.242949 28698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:12:57.243664 28724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:57.243832 28698 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:57.243920 28698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "7350151b5e834ac29b1bb35e605daead"
format_stamp: "Formatted at 2026-08-07 20:12:57 on dist-test-slave-2j7r"
server_key: "7b63c27fd8826d68fd0071f5a7fb697b"
server_key_iv: "393c6bd40a575e9332896b7614119e77"
server_key_version: "encryptionkey@0"
I20260807 20:12:57.243980 28698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:57.262442 28698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:57.264564 28698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:57.264700 28698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:57.264917 28698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:57.265267 28698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:57.265305 28698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:57.265336 28698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:57.265359 28698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:57.271193 28698 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:41199
I20260807 20:12:57.271274 28905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:41199 every 8 connection(s)
I20260807 20:12:57.271569 28698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
Aug 07 20:12:57 dist-test-slave-2j7r krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133577, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:57.279052 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 28698
I20260807 20:12:57.279143 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:12:57.279335 24355 external_mini_cluster.cc:1428] Setting key 5149e855f2a84742d72a5bdf8dd14351
I20260807 20:12:57.280593 28222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:57.272340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:32841 (local address 127.23.200.254:44503)
0807 20:12:57.272395 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:57.272397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:57.272596 (+ 199us) server_negotiation.cc:407] Connection header received
0807 20:12:57.273129 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:57.273131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:57.273144 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:57.273199 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:57.274185 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:57.274678 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:57.275651 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:57.275843 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:57.275906 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:57.277954 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:57.277956 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:57.277956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:57.277964 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:57.279072 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:57.279494 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:57.279496 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:57.279496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:57.279565 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:57.279838 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:57.279839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:57.279839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:57.280013 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:57.280064 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:57.280442 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 20:12:57.280510 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 20:12:57.280767 28908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:57.272295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:44503 (local address 127.23.200.195:32841)
0807 20:12:57.272477 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:57.272514 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:57.273009 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:57.273278 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:57.273283 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:57.273884 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:57.274085 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:57.274090 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:57.274762 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:57.274763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:57.275566 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:57.275569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:57.275912 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:57.275913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:57.276039 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:57.276490 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:57.276499 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:57.277861 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:57.279145 (+ 1284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:57.279152 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:57.279155 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:57.279380 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:57.279644 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:57.279646 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:57.279646 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:57.279765 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:57.280062 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:57.280069 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:57.280237 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 20:12:57.280407 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 20:12:57.280542 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
I20260807 20:12:57.281180 28906 heartbeater.cc:347] Connected to a master server at 127.23.200.254:44503
I20260807 20:12:57.281281 28906 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:57.281456 28906 heartbeater.cc:511] Master 127.23.200.254:44503 requested a full tablet report, sending...
I20260807 20:12:57.281870 28170 ts_manager.cc:194] Registered new tserver with Master: 7350151b5e834ac29b1bb35e605daead (127.23.200.195:41199)
I20260807 20:12:57.282243 28170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:32841
I20260807 20:12:57.291054 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 20:12:57.302975 28222 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:57.302092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39614 (local address 127.23.200.254:44503)
0807 20:12:57.302149 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:57.302151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:57.302269 (+ 118us) server_negotiation.cc:407] Connection header received
0807 20:12:57.302341 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:57.302342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:57.302353 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:57.302407 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:57.302915 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39614: BlockingRecv error: recv got EOF from 127.0.0.1:39614 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 20:12:57.303931 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 28268
I20260807 20:12:57.327273 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 28485
I20260807 20:12:57.337710 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 28698
I20260807 20:12:57.355192 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 28121
2026-08-07T20:12:57Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2040 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:57 dist-test-slave-2j7r krb5kdc[28928](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:57 dist-test-slave-2j7r krb5kdc[28928](info): set up 2 sockets
Aug 07 20:12:57 dist-test-slave-2j7r krb5kdc[28928](info): commencing operation
krb5kdc: starting...
W20260807 20:12:58.528406 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:12:58 dist-test-slave-2j7r krb5kdc[28928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:12:58.649888 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:44523
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:44523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:58.731776 28947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:58.731984 28947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:58.732013 28947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:58.733179 28947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:58.733222 28947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:58.733239 28947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:58.733255 28947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:58.733270 28947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:58.734772 28947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:44523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:44523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:58.735018 28947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:58.735222 28947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:58.740157 28955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:58.740208 28956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:58.740195 28959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:58.740239 28947 server_base.cc:1034] running on GCE node
I20260807 20:12:58.740549 28947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:58.740811 28947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:58.741959 28947 hybrid_clock.cc:648] HybridClock initialized: now 1786133578741944 us; error 32 us; skew 500 ppm
Aug 07 20:12:58 dist-test-slave-2j7r krb5kdc[28928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133578, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:58.744333 28947 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:12:58.744750 28947 webserver.cc:460] Webserver started at http://127.23.200.254:37127/ using document root <none> and password file <none>
I20260807 20:12:58.744938 28947 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:58.744987 28947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:58.745146 28947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:58.746189 28947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "5b131145edbe453799f8bc9d7a9ce6af"
format_stamp: "Formatted at 2026-08-07 20:12:58 on dist-test-slave-2j7r"
server_key: "6182c1e59998ec638eff7b9e9939d443"
server_key_iv: "cf854772f0e547aa8c5a03f2cc4d884c"
server_key_version: "encryptionkey@0"
I20260807 20:12:58.746513 28947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "5b131145edbe453799f8bc9d7a9ce6af"
format_stamp: "Formatted at 2026-08-07 20:12:58 on dist-test-slave-2j7r"
server_key: "6182c1e59998ec638eff7b9e9939d443"
server_key_iv: "cf854772f0e547aa8c5a03f2cc4d884c"
server_key_version: "encryptionkey@0"
I20260807 20:12:58.747712 28947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:12:58.748406 28973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:58.748591 28947 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:58.748687 28947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "5b131145edbe453799f8bc9d7a9ce6af"
format_stamp: "Formatted at 2026-08-07 20:12:58 on dist-test-slave-2j7r"
server_key: "6182c1e59998ec638eff7b9e9939d443"
server_key_iv: "cf854772f0e547aa8c5a03f2cc4d884c"
server_key_version: "encryptionkey@0"
I20260807 20:12:58.748780 28947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:58.756537 28947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:58.758558 28947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:58.758689 28947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:58.762414 28947 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:44523
I20260807 20:12:58.762465 29057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:44523 every 8 connection(s)
I20260807 20:12:58.762899 28947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:58.763558 29058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:58.764891 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 28947
I20260807 20:12:58.764990 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:12:58.765196 24355 external_mini_cluster.cc:1428] Setting key 4ba8ebcfb3b2c649a4d551b4b313fe69
I20260807 20:12:58.766120 29058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af: Bootstrap starting.
I20260807 20:12:58.766837 29058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:58.767122 29058 log.cc:824] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:58.767879 29058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af: No bootstrap required, opened a new log
I20260807 20:12:58.770256 29058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b131145edbe453799f8bc9d7a9ce6af" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44523 } }
I20260807 20:12:58.770411 29058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:58.770448 29058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b131145edbe453799f8bc9d7a9ce6af, State: Initialized, Role: FOLLOWER
I20260807 20:12:58.770617 29058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b131145edbe453799f8bc9d7a9ce6af" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44523 } }
I20260807 20:12:58.770725 29058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:58.770769 29058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:58.770823 29058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:58.771526 29058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b131145edbe453799f8bc9d7a9ce6af" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44523 } }
I20260807 20:12:58.771647 29058 leader_election.cc:302] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b131145edbe453799f8bc9d7a9ce6af; no voters:
I20260807 20:12:58.771838 29058 leader_election.cc:288] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:58.771903 29069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:58.772080 29069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [term 1 LEADER]: Becoming Leader. State: Replica: 5b131145edbe453799f8bc9d7a9ce6af, State: Running, Role: LEADER
I20260807 20:12:58.772090 29058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:58.772187 29069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b131145edbe453799f8bc9d7a9ce6af" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44523 } }
Aug 07 20:12:58 dist-test-slave-2j7r krb5kdc[28928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:12:58.772540 29070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b131145edbe453799f8bc9d7a9ce6af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b131145edbe453799f8bc9d7a9ce6af" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44523 } } }
I20260807 20:12:58.772620 29070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:58.772773 29070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b131145edbe453799f8bc9d7a9ce6af. Latest consensus state: current_term: 1 leader_uuid: "5b131145edbe453799f8bc9d7a9ce6af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b131145edbe453799f8bc9d7a9ce6af" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 44523 } } }
I20260807 20:12:58.772845 29070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:58.773316 29078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:58.773873 29078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:58.775472 29078 catalog_manager.cc:1347] Generated new cluster ID: e61f74ed07324dac96969c91425f4149
I20260807 20:12:58.775534 29078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:58.776855 29066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:58.765676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52508 (local address 127.23.200.254:44523)
0807 20:12:58.765900 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:58.765909 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:58.765942 (+ 33us) server_negotiation.cc:407] Connection header received
0807 20:12:58.766487 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:58.766498 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:58.766565 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:58.766714 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:58.767370 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:58.768403 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:58.769528 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:58.769928 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:58.770002 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:58.773224 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:58.773229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:58.773229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:58.773243 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:58.775240 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:58.775716 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:58.775720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:58.775720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:58.775831 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:58.776037 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:58.776038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:58.776039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:58.776289 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:58.776410 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:58.776526 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 20:12:58.776656 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":111,"threads_started":1}
I20260807 20:12:58.777724 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:58.783782 29078 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:58.784484 29078 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:58.792265 29078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b131145edbe453799f8bc9d7a9ce6af: Generated new TSK 0
I20260807 20:12:58.792469 29078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:58.798106 29066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:58.794348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52510 (local address 127.23.200.254:44523)
0807 20:12:58.794405 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:58.794407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:58.794527 (+ 120us) server_negotiation.cc:407] Connection header received
0807 20:12:58.794606 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:58.794607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:58.794617 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:58.794672 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:58.795101 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:58.795612 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:58.796688 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:58.796919 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:58.796981 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:58.797663 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:58.797664 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:58.797665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:58.797852 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:58.797870 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:58.797877 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:58.797938 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:58.798024 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 20:12:58.798049 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:12:58.800696 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 28947
2026-08-07T20:12:58Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1443 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:58 dist-test-slave-2j7r krb5kdc[29112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:58 dist-test-slave-2j7r krb5kdc[29112](info): set up 2 sockets
Aug 07 20:12:58 dist-test-slave-2j7r krb5kdc[29112](info): commencing operation
krb5kdc: starting...
W20260807 20:12:59.952944 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:13:00 dist-test-slave-2j7r krb5kdc[29112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:13:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:13:00.076409 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:42119
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:42119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:00.162945 29129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:00.163157 29129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:00.163179 29129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:00.164269 29129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:00.164301 29129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:00.164314 29129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:00.164324 29129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:00.164335 29129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:13:00.165817 29129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:42119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:42119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:00.166038 29129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:00.166219 29129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:00.170939 29136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:00.171041 29137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:00.170970 29142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:00.171185 29129 server_base.cc:1034] running on GCE node
I20260807 20:13:00.171452 29129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:00.171726 29129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:00.172873 29129 hybrid_clock.cc:648] HybridClock initialized: now 1786133580172855 us; error 32 us; skew 500 ppm
Aug 07 20:13:00 dist-test-slave-2j7r krb5kdc[29112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133580, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:00.175381 29129 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:13:00.175836 29129 webserver.cc:460] Webserver started at http://127.23.200.254:40617/ using document root <none> and password file <none>
I20260807 20:13:00.176036 29129 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:00.176083 29129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:00.176208 29129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:00.177371 29129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "08355f077564478dbad3a60a3d5a8352"
format_stamp: "Formatted at 2026-08-07 20:13:00 on dist-test-slave-2j7r"
server_key: "a7058e741b424cdf283c1329bb80e848"
server_key_iv: "c5f8c4da3a589a39af6e9be38ce78531"
server_key_version: "encryptionkey@0"
I20260807 20:13:00.177713 29129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "08355f077564478dbad3a60a3d5a8352"
format_stamp: "Formatted at 2026-08-07 20:13:00 on dist-test-slave-2j7r"
server_key: "a7058e741b424cdf283c1329bb80e848"
server_key_iv: "c5f8c4da3a589a39af6e9be38ce78531"
server_key_version: "encryptionkey@0"
I20260807 20:13:00.179013 29129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:13:00.179701 29153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:00.179863 29129 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:00.179956 29129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "08355f077564478dbad3a60a3d5a8352"
format_stamp: "Formatted at 2026-08-07 20:13:00 on dist-test-slave-2j7r"
server_key: "a7058e741b424cdf283c1329bb80e848"
server_key_iv: "c5f8c4da3a589a39af6e9be38ce78531"
server_key_version: "encryptionkey@0"
I20260807 20:13:00.180016 29129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:00.194739 29129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:00.196874 29129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:00.197024 29129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:00.200836 29129 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:42119
I20260807 20:13:00.200872 29239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:42119 every 8 connection(s)
I20260807 20:13:00.201277 29129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:13:00.201804 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 29129
I20260807 20:13:00.201932 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:13:00.202137 24355 external_mini_cluster.cc:1428] Setting key 8d2fa45e316866f50216390391aac262
I20260807 20:13:00.202565 29240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:00.205679 29240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352: Bootstrap starting.
I20260807 20:13:00.206377 29240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:00.206650 29240 log.cc:824] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:00.207338 29240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352: No bootstrap required, opened a new log
Aug 07 20:13:00 dist-test-slave-2j7r krb5kdc[29112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:00.209416 29240 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08355f077564478dbad3a60a3d5a8352" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42119 } }
I20260807 20:13:00.209568 29240 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:00.209589 29240 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08355f077564478dbad3a60a3d5a8352, State: Initialized, Role: FOLLOWER
I20260807 20:13:00.209748 29240 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08355f077564478dbad3a60a3d5a8352" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42119 } }
I20260807 20:13:00.209826 29240 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:13:00.209859 29240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:13:00.209909 29240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:00.210438 29240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08355f077564478dbad3a60a3d5a8352" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42119 } }
I20260807 20:13:00.210546 29240 leader_election.cc:302] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08355f077564478dbad3a60a3d5a8352; no voters:
I20260807 20:13:00.210718 29240 leader_election.cc:288] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:13:00.210768 29249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:00.210935 29249 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [term 1 LEADER]: Becoming Leader. State: Replica: 08355f077564478dbad3a60a3d5a8352, State: Running, Role: LEADER
I20260807 20:13:00.211006 29240 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:00.211064 29249 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08355f077564478dbad3a60a3d5a8352" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42119 } }
I20260807 20:13:00.212045 29251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08355f077564478dbad3a60a3d5a8352. Latest consensus state: current_term: 1 leader_uuid: "08355f077564478dbad3a60a3d5a8352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08355f077564478dbad3a60a3d5a8352" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42119 } } }
I20260807 20:13:00.212123 29251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:00.212322 29271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:13:00.212105 29250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08355f077564478dbad3a60a3d5a8352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08355f077564478dbad3a60a3d5a8352" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42119 } } }
I20260807 20:13:00.212467 29250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:00.212991 29271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:00.213235 29245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:00.202710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33138 (local address 127.23.200.254:42119)
0807 20:13:00.202871 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:00.202879 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:00.202907 (+ 28us) server_negotiation.cc:407] Connection header received
0807 20:13:00.203344 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:00.203351 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:00.203450 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:00.203568 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:00.204350 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:00.205299 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:00.206286 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:00.206736 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:00.206831 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:00.209231 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:00.209235 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:00.209235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:00.209247 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:00.211031 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:00.211491 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:00.211496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:00.211496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:00.211617 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:00.211831 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:00.211832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:00.211832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:00.212157 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:00.212312 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:00.212847 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 20:13:00.212987 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260807 20:13:00.214504 29271 catalog_manager.cc:1347] Generated new cluster ID: af3321f529d44f54b61947fb7313f0c5
I20260807 20:13:00.214560 29271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:13:00.223543 29271 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:13:00.224318 29271 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:00.236773 29271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08355f077564478dbad3a60a3d5a8352: Generated new TSK 0
I20260807 20:13:00.237039 29271 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:13:00.269505 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:42119
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:00.352290 29289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:00.352491 29289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:00.352521 29289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:00.353677 29289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:00.353717 29289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:00.353752 29289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:13:00.355198 29289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:42119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:00.355429 29289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:00.355646 29289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:00.360608 29297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:00.360646 29289 server_base.cc:1034] running on GCE node
W20260807 20:13:00.360608 29301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:00.360639 29299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:00.361064 29289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:00.361346 29289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:00.362483 29289 hybrid_clock.cc:648] HybridClock initialized: now 1786133580362470 us; error 29 us; skew 500 ppm
Aug 07 20:13:00 dist-test-slave-2j7r krb5kdc[29112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133580, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:00.365221 29289 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:13:00.365638 29289 webserver.cc:460] Webserver started at http://127.23.200.193:37485/ using document root <none> and password file <none>
I20260807 20:13:00.365819 29289 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:00.365856 29289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:00.365965 29289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:00.366986 29289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "4b7f9c6f478d4e0cbe23e2236e192d31"
format_stamp: "Formatted at 2026-08-07 20:13:00 on dist-test-slave-2j7r"
server_key: "25161193e03cbef254b8f758b36f26c0"
server_key_iv: "cae5de4b6d24a079da005d1b41003958"
server_key_version: "encryptionkey@0"
I20260807 20:13:00.367281 29289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "4b7f9c6f478d4e0cbe23e2236e192d31"
format_stamp: "Formatted at 2026-08-07 20:13:00 on dist-test-slave-2j7r"
server_key: "25161193e03cbef254b8f758b36f26c0"
server_key_iv: "cae5de4b6d24a079da005d1b41003958"
server_key_version: "encryptionkey@0"
I20260807 20:13:00.368600 29289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:13:00.369325 29311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:00.369488 29289 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:00.369583 29289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "4b7f9c6f478d4e0cbe23e2236e192d31"
format_stamp: "Formatted at 2026-08-07 20:13:00 on dist-test-slave-2j7r"
server_key: "25161193e03cbef254b8f758b36f26c0"
server_key_iv: "cae5de4b6d24a079da005d1b41003958"
server_key_version: "encryptionkey@0"
I20260807 20:13:00.369647 29289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:00.383227 29289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:00.385167 29289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:00.385304 29289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:00.385530 29289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:00.385838 29289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:00.385870 29289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:00.385900 29289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:00.385917 29289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:00.391520 29289 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:44469
I20260807 20:13:00.391585 29486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:44469 every 8 connection(s)
I20260807 20:13:00.391873 29289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:13:00.395460 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 29289
I20260807 20:13:00.395542 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:13:00.395709 24355 external_mini_cluster.cc:1428] Setting key 0f3c3bb9ca1694d87e92dd7299450cea
Aug 07 20:13:00 dist-test-slave-2j7r krb5kdc[29112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133580, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:00.401118 29245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:00.392565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:44647 (local address 127.23.200.254:42119)
0807 20:13:00.392627 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:00.392629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:00.392843 (+ 214us) server_negotiation.cc:407] Connection header received
0807 20:13:00.393412 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:00.393413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:00.393424 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:00.393483 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:00.394438 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:00.394944 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:00.395982 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:00.396189 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:00.396247 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:00.398361 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:00.398363 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:00.398363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:00.398373 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:00.399708 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:00.400090 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:00.400091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:00.400092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:00.400158 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:00.400385 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:00.400386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:00.400386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:00.400535 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:00.400605 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:00.400886 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 20:13:00.400991 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:13:00.401180 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:00.392537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42119 (local address 127.23.200.193:44647)
0807 20:13:00.392711 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:00.392751 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:00.393291 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:00.393541 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:00.393544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:00.394145 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:00.394344 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:00.394349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:00.395015 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:00.395015 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:00.395898 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:00.395901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:00.396260 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:00.396261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:00.396396 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:00.396850 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:00.396859 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:00.398270 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:00.399784 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:00.399790 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:00.399792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:00.400022 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:00.400218 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:00.400218 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:00.400218 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:00.400319 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:00.400600 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:00.400604 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:00.400758 (+ 154us) client_negotiation.cc:769] Sending connection context
0807 20:13:00.400845 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 20:13:00.400970 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260807 20:13:00.401913 29487 heartbeater.cc:347] Connected to a master server at 127.23.200.254:42119
I20260807 20:13:00.402009 29487 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:00.402179 29487 heartbeater.cc:511] Master 127.23.200.254:42119 requested a full tablet report, sending...
I20260807 20:13:00.402683 29183 ts_manager.cc:194] Registered new tserver with Master: 4b7f9c6f478d4e0cbe23e2236e192d31 (127.23.200.193:44469)
I20260807 20:13:00.403270 29183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:44647
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:13:00.423632 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:42119
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:00.505702 29493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:00.505906 29493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:00.505939 29493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:00.507081 29493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:00.507123 29493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:00.507156 29493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:13:00.508592 29493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:42119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:00.508829 29493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:00.509043 29493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:00.514083 29504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:00.514087 29502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:00.514281 29493 server_base.cc:1034] running on GCE node
W20260807 20:13:00.514166 29500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:00.514518 29493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:00.514740 29493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:00.515869 29493 hybrid_clock.cc:648] HybridClock initialized: now 1786133580515856 us; error 29 us; skew 500 ppm
Aug 07 20:13:00 dist-test-slave-2j7r krb5kdc[29112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133580, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:00.518388 29493 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:13:00.518783 29493 webserver.cc:460] Webserver started at http://127.23.200.194:44945/ using document root <none> and password file <none>
I20260807 20:13:00.519013 29493 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:00.519063 29493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:00.519191 29493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:00.520197 29493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "d5f403b146644e388dda97b28457ddb9"
format_stamp: "Formatted at 2026-08-07 20:13:00 on dist-test-slave-2j7r"
server_key: "6745806a15e1d2bf254512031d7cde22"
server_key_iv: "c7499cb3437e55b3408e05a79113b9f2"
server_key_version: "encryptionkey@0"
I20260807 20:13:00.520530 29493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "d5f403b146644e388dda97b28457ddb9"
format_stamp: "Formatted at 2026-08-07 20:13:00 on dist-test-slave-2j7r"
server_key: "6745806a15e1d2bf254512031d7cde22"
server_key_iv: "c7499cb3437e55b3408e05a79113b9f2"
server_key_version: "encryptionkey@0"
I20260807 20:13:00.521826 29493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:13:00.522488 29518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:00.522642 29493 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:13:00.522727 29493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "d5f403b146644e388dda97b28457ddb9"
format_stamp: "Formatted at 2026-08-07 20:13:00 on dist-test-slave-2j7r"
server_key: "6745806a15e1d2bf254512031d7cde22"
server_key_iv: "c7499cb3437e55b3408e05a79113b9f2"
server_key_version: "encryptionkey@0"
I20260807 20:13:00.522784 29493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:00.549541 29493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:00.551733 29493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:00.551878 29493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:00.552117 29493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:00.552419 29493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:00.552452 29493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:00.552481 29493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:00.552505 29493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:00.558136 29493 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:39527
I20260807 20:13:00.558192 29702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:39527 every 8 connection(s)
I20260807 20:13:00.558494 29493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:13:00.559079 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 29493
I20260807 20:13:00.559182 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:13:00.559365 24355 external_mini_cluster.cc:1428] Setting key 4d6faa403fcbf8950f6f38293756f408
Aug 07 20:13:00 dist-test-slave-2j7r krb5kdc[29112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133580, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
I20260807 20:13:00.568776 29245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:00.559197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:47329 (local address 127.23.200.254:42119)
0807 20:13:00.559288 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:00.559290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:00.559453 (+ 163us) server_negotiation.cc:407] Connection header received
0807 20:13:00.559988 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:00.559989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:00.560001 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:00.560057 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:00.562449 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:00.562947 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:00.563930 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:00.564150 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:00.564208 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:00.566291 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:00.566293 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:00.566293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:00.566300 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:00.567505 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:00.567879 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:00.567881 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:00.567881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:00.567951 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:00.568171 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:00.568172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:00.568173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:00.568280 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:00.568340 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:00.568599 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 20:13:00.568683 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 20:13:00.568940 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:00.559137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42119 (local address 127.23.200.194:47329)
0807 20:13:00.559320 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:00.559367 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:00.559879 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:00.561255 (+ 1376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:00.561260 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:00.562039 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:00.562319 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:00.562326 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:00.563050 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:00.563051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:00.563847 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:00.563849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:00.564224 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:00.564225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:00.564341 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:00.564826 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:00.564835 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:00.566204 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:00.567576 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:00.567583 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:00.567585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:00.567815 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:00.568010 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:00.568011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:00.568011 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:00.568117 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:00.568335 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:00.568342 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:00.568483 (+ 141us) client_negotiation.cc:769] Sending connection context
0807 20:13:00.568571 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 20:13:00.568724 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
I20260807 20:13:00.569413 29704 heartbeater.cc:347] Connected to a master server at 127.23.200.254:42119
I20260807 20:13:00.569501 29704 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:00.569653 29704 heartbeater.cc:511] Master 127.23.200.254:42119 requested a full tablet report, sending...
I20260807 20:13:00.569998 29183 ts_manager.cc:194] Registered new tserver with Master: d5f403b146644e388dda97b28457ddb9 (127.23.200.194:39527)
I20260807 20:13:00.570394 29183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:47329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:13:00.586238 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:42119
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:00.668087 29713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:00.668286 29713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:00.668309 29713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:00.669492 29713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:00.669538 29713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:00.669564 29713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:13:00.671062 29713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:42119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:00.671288 29713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:00.671501 29713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:00.676357 29727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:00.676548 29713 server_base.cc:1034] running on GCE node
W20260807 20:13:00.676337 29726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:00.676347 29729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:00.676795 29713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:00.677012 29713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:00.678148 29713 hybrid_clock.cc:648] HybridClock initialized: now 1786133580678134 us; error 26 us; skew 500 ppm
Aug 07 20:13:00 dist-test-slave-2j7r krb5kdc[29112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133580, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:00.680532 29713 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:13:00.680961 29713 webserver.cc:460] Webserver started at http://127.23.200.195:39223/ using document root <none> and password file <none>
I20260807 20:13:00.681243 29713 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:00.681296 29713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:00.681427 29713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:00.682477 29713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "465e7ddc9bbd4dc3a9445e25b64b7845"
format_stamp: "Formatted at 2026-08-07 20:13:00 on dist-test-slave-2j7r"
server_key: "0f75b1ec63a9e894495fa274d8457c96"
server_key_iv: "02158f6d3c9d6a28e3b142c15317d9c3"
server_key_version: "encryptionkey@0"
I20260807 20:13:00.682811 29713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "465e7ddc9bbd4dc3a9445e25b64b7845"
format_stamp: "Formatted at 2026-08-07 20:13:00 on dist-test-slave-2j7r"
server_key: "0f75b1ec63a9e894495fa274d8457c96"
server_key_iv: "02158f6d3c9d6a28e3b142c15317d9c3"
server_key_version: "encryptionkey@0"
I20260807 20:13:00.684134 29713 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:13:00.684859 29741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:00.685019 29713 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:00.685127 29713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "465e7ddc9bbd4dc3a9445e25b64b7845"
format_stamp: "Formatted at 2026-08-07 20:13:00 on dist-test-slave-2j7r"
server_key: "0f75b1ec63a9e894495fa274d8457c96"
server_key_iv: "02158f6d3c9d6a28e3b142c15317d9c3"
server_key_version: "encryptionkey@0"
I20260807 20:13:00.685191 29713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:00.697650 29713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:00.699828 29713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:00.699980 29713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:00.700204 29713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:00.700521 29713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:00.700555 29713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:00.700584 29713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:00.700608 29713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:00.706154 29713 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:42667
I20260807 20:13:00.706209 29921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:42667 every 8 connection(s)
I20260807 20:13:00.706522 29713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:13:00.711632 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 29713
I20260807 20:13:00.711737 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:13:00.711937 24355 external_mini_cluster.cc:1428] Setting key 255f9bc64983c2be6375885ef26f56bc
Aug 07 20:13:00 dist-test-slave-2j7r krb5kdc[29112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133580, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:00.715574 29245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:00.707192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:36979 (local address 127.23.200.254:42119)
0807 20:13:00.707250 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:00.707252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:00.707482 (+ 230us) server_negotiation.cc:407] Connection header received
0807 20:13:00.707993 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:00.707995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:00.708008 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:00.708060 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:00.709003 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:00.709550 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:00.710541 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:00.710785 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:00.710842 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:00.713357 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:00.713359 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:00.713359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:00.713371 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:00.714476 (+ 1105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:00.714836 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:00.714837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:00.714838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:00.714898 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:00.715074 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:00.715075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:00.715075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:00.715172 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:00.715218 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:00.715433 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 20:13:00.715508 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:13:00.715709 29926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:00.707193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42119 (local address 127.23.200.195:36979)
0807 20:13:00.707355 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:00.707391 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:00.707889 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:00.708135 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:00.708140 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:00.708708 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:00.708901 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:00.708907 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:00.709634 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:00.709635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:00.710450 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:00.710452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:00.710859 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:00.710860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:00.710993 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:00.711480 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:00.711491 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:00.713246 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:00.714539 (+ 1293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:00.714545 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:00.714547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:00.714757 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:00.714952 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:00.714952 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:00.714952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:00.715040 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:00.715211 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:00.715214 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:00.715324 (+ 110us) client_negotiation.cc:769] Sending connection context
0807 20:13:00.715400 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 20:13:00.715519 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260807 20:13:00.716306 29923 heartbeater.cc:347] Connected to a master server at 127.23.200.254:42119
I20260807 20:13:00.716394 29923 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:00.716590 29923 heartbeater.cc:511] Master 127.23.200.254:42119 requested a full tablet report, sending...
I20260807 20:13:00.716974 29183 ts_manager.cc:194] Registered new tserver with Master: 465e7ddc9bbd4dc3a9445e25b64b7845 (127.23.200.195:42667)
I20260807 20:13:00.717350 29183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:36979
I20260807 20:13:00.723589 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 20:13:00 dist-test-slave-2j7r krb5kdc[29112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133580, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 20:13:00 dist-test-slave-2j7r krb5kdc[29112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133580, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:00.746238 29245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:00.739983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33140 (local address 127.23.200.254:42119)
0807 20:13:00.740043 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:00.740046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:00.740119 (+ 73us) server_negotiation.cc:407] Connection header received
0807 20:13:00.740169 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:00.740170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:00.740180 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:00.740238 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:00.740741 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:00.741282 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:00.741958 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:00.742130 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:00.742171 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:00.744237 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:00.744239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:00.744239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:00.744247 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:00.745354 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:00.745646 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:00.745646 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:00.745646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:00.745692 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:00.745817 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:00.745817 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:00.745818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:00.745973 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:00.746047 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:00.746115 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 20:13:00.746171 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260807 20:13:00.746472 29183 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33140
I20260807 20:13:00.746847 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 29289
I20260807 20:13:00.772054 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 29493
I20260807 20:13:00.784962 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 29713
I20260807 20:13:00.800843 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 29129
2026-08-07T20:13:00Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1997 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:13:00 dist-test-slave-2j7r krb5kdc[29944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:13:00 dist-test-slave-2j7r krb5kdc[29944](info): set up 2 sockets
Aug 07 20:13:00 dist-test-slave-2j7r krb5kdc[29944](info): commencing operation
krb5kdc: starting...
W20260807 20:13:01.963212 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:13:02 dist-test-slave-2j7r krb5kdc[29944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:13:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:13:02.084251 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:39095
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:40337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:39095
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:02.167378 29966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:02.167600 29966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:02.167629 29966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:02.168761 29966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:02.168800 29966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:02.168817 29966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:02.168834 29966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:02.168849 29966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:13:02.170401 29966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:40337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:39095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:39095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:02.170655 29966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:02.170863 29966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:02.175806 29975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:02.175809 29974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:02.175894 29966 server_base.cc:1034] running on GCE node
W20260807 20:13:02.175809 29978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:02.176257 29966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:02.176533 29966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:02.177671 29966 hybrid_clock.cc:648] HybridClock initialized: now 1786133582177657 us; error 30 us; skew 500 ppm
Aug 07 20:13:02 dist-test-slave-2j7r krb5kdc[29944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133582, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:02.180143 29966 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:13:02.180575 29966 webserver.cc:460] Webserver started at http://127.23.200.254:44809/ using document root <none> and password file <none>
I20260807 20:13:02.180771 29966 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:02.180812 29966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:02.180948 29966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:02.182041 29966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "db5e259c77ff46ffacdc215bb0c746d0"
format_stamp: "Formatted at 2026-08-07 20:13:02 on dist-test-slave-2j7r"
server_key: "b2332f3a76cc5c3c517462570910574e"
server_key_iv: "c7bbfea66dc3c3d15f4b7dd3956ad638"
server_key_version: "encryptionkey@0"
I20260807 20:13:02.182376 29966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "db5e259c77ff46ffacdc215bb0c746d0"
format_stamp: "Formatted at 2026-08-07 20:13:02 on dist-test-slave-2j7r"
server_key: "b2332f3a76cc5c3c517462570910574e"
server_key_iv: "c7bbfea66dc3c3d15f4b7dd3956ad638"
server_key_version: "encryptionkey@0"
I20260807 20:13:02.183650 29966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:13:02.184371 29991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:02.184542 29966 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:02.184626 29966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "db5e259c77ff46ffacdc215bb0c746d0"
format_stamp: "Formatted at 2026-08-07 20:13:02 on dist-test-slave-2j7r"
server_key: "b2332f3a76cc5c3c517462570910574e"
server_key_iv: "c7bbfea66dc3c3d15f4b7dd3956ad638"
server_key_version: "encryptionkey@0"
I20260807 20:13:02.184696 29966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:02.208798 29966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:02.211133 29966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:02.211269 29966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:02.215384 29966 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:39095
I20260807 20:13:02.215444 30086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:39095 every 8 connection(s)
I20260807 20:13:02.215926 29966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:13:02.216632 30087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:02.219012 30087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0: Bootstrap starting.
I20260807 20:13:02.219635 30087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:02.219769 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 29966
I20260807 20:13:02.219873 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:13:02.219981 30087 log.cc:824] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:02.220074 24355 external_mini_cluster.cc:1428] Setting key 981905105ce676167b5e487d233a7d64
I20260807 20:13:02.220818 30087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0: No bootstrap required, opened a new log
I20260807 20:13:02.223556 30087 raft_consensus.cc:348] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5e259c77ff46ffacdc215bb0c746d0" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39095 } }
I20260807 20:13:02.223750 30087 raft_consensus.cc:374] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:02.223793 30087 raft_consensus.cc:729] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db5e259c77ff46ffacdc215bb0c746d0, State: Initialized, Role: FOLLOWER
I20260807 20:13:02.223953 30087 consensus_queue.cc:260] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5e259c77ff46ffacdc215bb0c746d0" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39095 } }
I20260807 20:13:02.224038 30087 raft_consensus.cc:388] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:13:02.224063 30087 raft_consensus.cc:482] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:13:02.224112 30087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:02.224730 30087 raft_consensus.cc:504] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5e259c77ff46ffacdc215bb0c746d0" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39095 } }
I20260807 20:13:02.224838 30087 leader_election.cc:302] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db5e259c77ff46ffacdc215bb0c746d0; no voters:
I20260807 20:13:02.225028 30087 leader_election.cc:288] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:13:02.225080 30097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:02.225322 30087 sys_catalog.cc:564] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:02.225358 30097 raft_consensus.cc:686] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [term 1 LEADER]: Becoming Leader. State: Replica: db5e259c77ff46ffacdc215bb0c746d0, State: Running, Role: LEADER
I20260807 20:13:02.225521 30097 consensus_queue.cc:237] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5e259c77ff46ffacdc215bb0c746d0" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39095 } }
I20260807 20:13:02.226840 30098 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db5e259c77ff46ffacdc215bb0c746d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5e259c77ff46ffacdc215bb0c746d0" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39095 } } }
I20260807 20:13:02.226995 30098 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:02.227079 30097 sys_catalog.cc:455] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db5e259c77ff46ffacdc215bb0c746d0. Latest consensus state: current_term: 1 leader_uuid: "db5e259c77ff46ffacdc215bb0c746d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db5e259c77ff46ffacdc215bb0c746d0" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39095 } } }
I20260807 20:13:02.227161 30097 sys_catalog.cc:458] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:02.227331 30108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 07 20:13:02 dist-test-slave-2j7r krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:02.227843 30108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:02.229445 30108 catalog_manager.cc:1347] Generated new cluster ID: c54a0b4d2a6843e495204ef59c0a6a96
I20260807 20:13:02.229929 30108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:13:02.231171 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:02.220652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38964 (local address 127.23.200.254:39095)
0807 20:13:02.220835 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:02.220845 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:02.220881 (+ 36us) server_negotiation.cc:407] Connection header received
0807 20:13:02.221425 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:02.221435 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:02.221503 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:02.221667 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:02.222281 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.223655 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.224609 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.225253 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.225418 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:02.227856 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:02.227861 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:02.227861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:02.227876 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:02.229701 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:02.230058 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:02.230061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:02.230062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:02.230151 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:02.230331 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:02.230332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:02.230332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:02.230569 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:02.230699 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:02.230815 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 20:13:02.230961 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
I20260807 20:13:02.270704 30108 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:13:02.271463 30108 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:02.274387 30108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db5e259c77ff46ffacdc215bb0c746d0: Generated new TSK 0
I20260807 20:13:02.274566 30108 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:13:02.307253 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39095
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:40337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:02.390740 30125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:02.390959 30125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:02.390990 30125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:02.392199 30125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:02.392254 30125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:02.392290 30125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:13:02.393898 30125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:40337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:02.394143 30125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:02.394371 30125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:02.399550 30130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:02.399557 30133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:02.399618 30125 server_base.cc:1034] running on GCE node
W20260807 20:13:02.399608 30131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:02.399959 30125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:02.400190 30125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:02.401371 30125 hybrid_clock.cc:648] HybridClock initialized: now 1786133582401353 us; error 35 us; skew 500 ppm
Aug 07 20:13:02 dist-test-slave-2j7r krb5kdc[29944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133582, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:02.403887 30125 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:13:02.404371 30125 webserver.cc:460] Webserver started at http://127.23.200.193:44071/ using document root <none> and password file <none>
I20260807 20:13:02.404578 30125 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:02.404623 30125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:02.404729 30125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:02.405807 30125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "47b6fe893661456994e4a3abd4061969"
format_stamp: "Formatted at 2026-08-07 20:13:02 on dist-test-slave-2j7r"
server_key: "4ff0249100a88fbbcb3d5a0801845a2e"
server_key_iv: "fa6d0482e5eb740069f18e5835569f00"
server_key_version: "encryptionkey@0"
I20260807 20:13:02.406143 30125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "47b6fe893661456994e4a3abd4061969"
format_stamp: "Formatted at 2026-08-07 20:13:02 on dist-test-slave-2j7r"
server_key: "4ff0249100a88fbbcb3d5a0801845a2e"
server_key_iv: "fa6d0482e5eb740069f18e5835569f00"
server_key_version: "encryptionkey@0"
I20260807 20:13:02.407485 30125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:13:02.408190 30153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:02.408370 30125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:02.408455 30125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "47b6fe893661456994e4a3abd4061969"
format_stamp: "Formatted at 2026-08-07 20:13:02 on dist-test-slave-2j7r"
server_key: "4ff0249100a88fbbcb3d5a0801845a2e"
server_key_iv: "fa6d0482e5eb740069f18e5835569f00"
server_key_version: "encryptionkey@0"
I20260807 20:13:02.408511 30125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:02.423674 30125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:02.425567 30125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:02.425766 30125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:02.426019 30125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:02.426394 30125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:02.426451 30125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:02.426489 30125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:02.426535 30125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:02.432873 30125 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:44145
I20260807 20:13:02.432924 30340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:44145 every 8 connection(s)
I20260807 20:13:02.433288 30125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
Aug 07 20:13:02 dist-test-slave-2j7r krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133582, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:02.441821 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:02.433989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:35935 (local address 127.23.200.254:39095)
0807 20:13:02.434050 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:02.434053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:02.434248 (+ 195us) server_negotiation.cc:407] Connection header received
0807 20:13:02.435145 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:02.435146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:02.435156 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:02.435206 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:02.435779 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.436266 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.437272 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.437532 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.437583 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:02.439467 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:02.439469 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:02.439469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:02.439476 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:02.440713 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:02.441045 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:02.441046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:02.441046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:02.441121 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:02.441312 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:02.441313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:02.441313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:02.441437 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:02.441486 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:02.441688 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 20:13:02.441753 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:13:02.442013 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:02.433969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39095 (local address 127.23.200.193:35935)
0807 20:13:02.434134 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:02.434171 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:02.435058 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:02.435263 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:02.435266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:02.435524 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:02.435714 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:02.435718 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.436336 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.436337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:02.437196 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:02.437198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.437591 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.437592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:02.437689 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:02.438095 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:02.438103 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:02.439384 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:02.440776 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:02.440781 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:02.440782 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:02.440990 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:02.441166 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:02.441166 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:02.441166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:02.441266 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:02.441483 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:02.441486 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:02.441593 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 20:13:02.441669 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 20:13:02.441790 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260807 20:13:02.442528 30341 heartbeater.cc:347] Connected to a master server at 127.23.200.254:39095
I20260807 20:13:02.442620 30341 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:02.442808 30341 heartbeater.cc:511] Master 127.23.200.254:39095 requested a full tablet report, sending...
I20260807 20:13:02.443253 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 30125
I20260807 20:13:02.443356 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:13:02.443348 30019 ts_manager.cc:194] Registered new tserver with Master: 47b6fe893661456994e4a3abd4061969 (127.23.200.193:44145)
I20260807 20:13:02.443562 24355 external_mini_cluster.cc:1428] Setting key 65da0ebb2a82a591e11770222bae7004
I20260807 20:13:02.444098 30019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:35935
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:13:02.469851 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39095
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:40337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:02.552155 30347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:02.552381 30347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:02.552413 30347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:02.553596 30347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:02.553637 30347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:02.553663 30347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:13:02.555147 30347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:40337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:02.555341 30347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:02.555536 30347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:02.560308 30363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:02.560317 30361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:02.560313 30359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:02.560568 30347 server_base.cc:1034] running on GCE node
I20260807 20:13:02.560828 30347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:02.561054 30347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:02.562222 30347 hybrid_clock.cc:648] HybridClock initialized: now 1786133582562210 us; error 27 us; skew 500 ppm
Aug 07 20:13:02 dist-test-slave-2j7r krb5kdc[29944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133582, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:02.564759 30347 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:13:02.565261 30347 webserver.cc:460] Webserver started at http://127.23.200.194:43973/ using document root <none> and password file <none>
I20260807 20:13:02.565485 30347 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:02.565532 30347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:02.565661 30347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:02.566730 30347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "95fa3629bd4348a0a4dd9907c9decc33"
format_stamp: "Formatted at 2026-08-07 20:13:02 on dist-test-slave-2j7r"
server_key: "40bbea5770cddcbe44af62828987940c"
server_key_iv: "d2022ac0f389fc6dfc8b3c7d4248eb6c"
server_key_version: "encryptionkey@0"
I20260807 20:13:02.567072 30347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "95fa3629bd4348a0a4dd9907c9decc33"
format_stamp: "Formatted at 2026-08-07 20:13:02 on dist-test-slave-2j7r"
server_key: "40bbea5770cddcbe44af62828987940c"
server_key_iv: "d2022ac0f389fc6dfc8b3c7d4248eb6c"
server_key_version: "encryptionkey@0"
I20260807 20:13:02.568370 30347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:13:02.569157 30374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:02.569304 30347 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:02.569442 30347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "95fa3629bd4348a0a4dd9907c9decc33"
format_stamp: "Formatted at 2026-08-07 20:13:02 on dist-test-slave-2j7r"
server_key: "40bbea5770cddcbe44af62828987940c"
server_key_iv: "d2022ac0f389fc6dfc8b3c7d4248eb6c"
server_key_version: "encryptionkey@0"
I20260807 20:13:02.569511 30347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:02.583758 30347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:02.585577 30347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:02.585709 30347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:02.585929 30347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:02.586249 30347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:02.586283 30347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:02.586313 30347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:02.586333 30347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:02.591939 30347 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:45711
I20260807 20:13:02.592007 30563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:45711 every 8 connection(s)
I20260807 20:13:02.592347 30347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:13:02.595391 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 30347
I20260807 20:13:02.595495 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:13:02.595661 24355 external_mini_cluster.cc:1428] Setting key 6a91c07d5ae7f6946e8548a8a3adbe26
Aug 07 20:13:02 dist-test-slave-2j7r krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133582, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:02.602454 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:02.593077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:41481 (local address 127.23.200.254:39095)
0807 20:13:02.593154 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:02.593156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:02.593354 (+ 198us) server_negotiation.cc:407] Connection header received
0807 20:13:02.594402 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:02.594404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:02.594416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:02.594481 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:02.595157 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.595795 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.596804 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.597013 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.597071 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:02.599476 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:02.599477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:02.599478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:02.599487 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:02.601007 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:02.601483 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:02.601484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:02.601485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:02.601551 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:02.601800 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:02.601801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:02.601801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:02.601948 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:02.602017 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:02.602258 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 20:13:02.602347 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 20:13:02.602546 30567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:02.593042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39095 (local address 127.23.200.194:41481)
0807 20:13:02.593226 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:02.593271 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:02.594265 (+ 994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:02.594548 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:02.594552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:02.594817 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:02.595068 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:02.595073 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.595894 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.595895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:02.596713 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:02.596715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.597115 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.597116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:02.597253 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:02.597724 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:02.597733 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:02.599382 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:02.601114 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:02.601123 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:02.601126 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:02.601376 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:02.601621 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:02.601621 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:02.601621 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:02.601731 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:02.602004 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:02.602007 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:02.602128 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 20:13:02.602204 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 20:13:02.602332 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
I20260807 20:13:02.603156 30565 heartbeater.cc:347] Connected to a master server at 127.23.200.254:39095
I20260807 20:13:02.603258 30565 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:02.603434 30565 heartbeater.cc:511] Master 127.23.200.254:39095 requested a full tablet report, sending...
I20260807 20:13:02.603817 30019 ts_manager.cc:194] Registered new tserver with Master: 95fa3629bd4348a0a4dd9907c9decc33 (127.23.200.194:45711)
I20260807 20:13:02.604189 30019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:41481
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:13:02.623605 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39095
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:40337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:02.706028 30570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:02.706233 30570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:02.706254 30570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:02.707365 30570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:02.707405 30570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:02.707430 30570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:13:02.708897 30570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:40337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:02.709151 30570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:02.709344 30570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:02.714401 30578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:02.714452 30576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:02.714453 30580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:02.714541 30570 server_base.cc:1034] running on GCE node
I20260807 20:13:02.714843 30570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:02.715070 30570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:02.716239 30570 hybrid_clock.cc:648] HybridClock initialized: now 1786133582716212 us; error 41 us; skew 500 ppm
Aug 07 20:13:02 dist-test-slave-2j7r krb5kdc[29944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133582, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:02.718747 30570 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:13:02.719148 30570 webserver.cc:460] Webserver started at http://127.23.200.195:45381/ using document root <none> and password file <none>
I20260807 20:13:02.719352 30570 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:02.719390 30570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:02.719481 30570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:02.720498 30570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "b3e04bed5a2e4371a17242e6639c9c18"
format_stamp: "Formatted at 2026-08-07 20:13:02 on dist-test-slave-2j7r"
server_key: "c09889c54f100864dbcbf791767a0a04"
server_key_iv: "d4f49101fdc468c73cf2936b4c89e339"
server_key_version: "encryptionkey@0"
I20260807 20:13:02.720794 30570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "b3e04bed5a2e4371a17242e6639c9c18"
format_stamp: "Formatted at 2026-08-07 20:13:02 on dist-test-slave-2j7r"
server_key: "c09889c54f100864dbcbf791767a0a04"
server_key_iv: "d4f49101fdc468c73cf2936b4c89e339"
server_key_version: "encryptionkey@0"
I20260807 20:13:02.722090 30570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:13:02.722769 30589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:02.722918 30570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:13:02.723004 30570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "b3e04bed5a2e4371a17242e6639c9c18"
format_stamp: "Formatted at 2026-08-07 20:13:02 on dist-test-slave-2j7r"
server_key: "c09889c54f100864dbcbf791767a0a04"
server_key_iv: "d4f49101fdc468c73cf2936b4c89e339"
server_key_version: "encryptionkey@0"
I20260807 20:13:02.723062 30570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:02.754822 30570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:02.756706 30570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:02.756829 30570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:02.757056 30570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:02.757421 30570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:02.757457 30570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:02.757478 30570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:02.757489 30570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:02.763036 30570 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:39385
I20260807 20:13:02.763111 30774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:39385 every 8 connection(s)
I20260807 20:13:02.763417 30570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:13:02.769274 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 30570
I20260807 20:13:02.769366 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:13:02.769560 24355 external_mini_cluster.cc:1428] Setting key eab2a3ef653a224ef1e1ddbb5c50202e
Aug 07 20:13:02 dist-test-slave-2j7r krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133582, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:02.772709 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:02.764216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:42691 (local address 127.23.200.254:39095)
0807 20:13:02.764336 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:02.764339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:02.764547 (+ 208us) server_negotiation.cc:407] Connection header received
0807 20:13:02.765524 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:02.765526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:02.765540 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:02.765611 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:02.766252 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.766744 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.767695 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.767884 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.767942 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:02.770202 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:02.770203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:02.770204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:02.770211 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:02.771519 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:02.771935 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:02.771936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:02.771937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:02.771994 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:02.772195 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:02.772196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:02.772196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:02.772300 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:02.772349 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:02.772583 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 20:13:02.772644 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 20:13:02.772967 30778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:02.764170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39095 (local address 127.23.200.195:42691)
0807 20:13:02.764352 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:02.764425 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:02.765379 (+ 954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:02.765684 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:02.765688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:02.765979 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:02.766185 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:02.766190 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.766809 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.766810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:02.767615 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:02.767618 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.767980 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.767981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:02.768114 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:02.768630 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:02.768639 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:02.770126 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:02.771598 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:02.771604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:02.771606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:02.771853 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:02.772044 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:02.772044 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:02.772045 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:02.772154 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:02.772348 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:02.772351 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:02.772465 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 20:13:02.772558 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 20:13:02.772742 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260807 20:13:02.773530 30776 heartbeater.cc:347] Connected to a master server at 127.23.200.254:39095
I20260807 20:13:02.773629 30776 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:02.773784 30776 heartbeater.cc:511] Master 127.23.200.254:39095 requested a full tablet report, sending...
I20260807 20:13:02.774121 30019 ts_manager.cc:194] Registered new tserver with Master: b3e04bed5a2e4371a17242e6639c9c18 (127.23.200.195:39385)
I20260807 20:13:02.774551 30019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:42691
I20260807 20:13:02.781037 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 20:13:02 dist-test-slave-2j7r krb5kdc[29944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 20:13:02 dist-test-slave-2j7r krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.193@KRBTEST.COM
I20260807 20:13:02.804683 30784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:02.797303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47448 (local address 127.23.200.193:44145)
0807 20:13:02.797515 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:02.797521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:02.797532 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:13:02.797558 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:02.797560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:02.797575 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:02.797642 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:02.798175 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.798882 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.799575 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.799898 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.799949 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:02.802081 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:02.802091 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:02.802091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:02.802098 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:02.803557 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:02.803878 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:02.803882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:02.803882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:02.803937 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:02.804106 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:02.804107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:02.804107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:02.804246 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:02.804383 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:02.804481 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 20:13:02.804560 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":116,"threads_started":1}
Aug 07 20:13:02 dist-test-slave-2j7r krb5kdc[29944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:02.812520 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:02.806076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38974 (local address 127.23.200.254:39095)
0807 20:13:02.806123 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:02.806125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:02.806250 (+ 125us) server_negotiation.cc:407] Connection header received
0807 20:13:02.806278 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:02.806279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:02.806290 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:02.806390 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:02.806825 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.807350 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.808005 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:02.808203 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:02.808264 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:02.810290 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:02.810292 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:02.810292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:02.810299 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:02.811588 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:02.811871 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:02.811872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:02.811872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:02.811929 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:02.812059 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:02.812059 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:02.812059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:02.812195 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:02.812266 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:02.812372 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 20:13:02.812433 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 20:13:02.812741 30019 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38974
I20260807 20:13:02.813045 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 30125
I20260807 20:13:02.829833 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 30347
I20260807 20:13:02.837814 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 30570
I20260807 20:13:02.847154 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 29966
2026-08-07T20:13:02Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2045 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:13:02 dist-test-slave-2j7r krb5kdc[30794](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:13:02 dist-test-slave-2j7r krb5kdc[30794](info): set up 2 sockets
Aug 07 20:13:02 dist-test-slave-2j7r krb5kdc[30794](info): commencing operation
krb5kdc: starting...
W20260807 20:13:04.013408 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:13:04 dist-test-slave-2j7r krb5kdc[30794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:13:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:13:04.135807 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:40209
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:37955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:40209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:04.218294 30811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:04.218524 30811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:04.218546 30811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:04.219683 30811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:04.219717 30811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:04.219729 30811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:04.219743 30811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:04.219754 30811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:13:04.221314 30811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:37955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:40209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:40209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:04.221542 30811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:04.221735 30811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:04.227016 30818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:04.227041 30811 server_base.cc:1034] running on GCE node
W20260807 20:13:04.227025 30817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:04.227025 30821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:04.227483 30811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:04.227802 30811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:04.228971 30811 hybrid_clock.cc:648] HybridClock initialized: now 1786133584228950 us; error 38 us; skew 500 ppm
Aug 07 20:13:04 dist-test-slave-2j7r krb5kdc[30794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133584, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:04.231552 30811 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:13:04.231866 30811 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:04.231920 30811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:04.232016 30811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:04.233076 30811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "924d4b7598bc412bb671cadbbc4ccee6"
format_stamp: "Formatted at 2026-08-07 20:13:04 on dist-test-slave-2j7r"
server_key: "b33356418eb8a4c57fc9d118c5f8188c"
server_key_iv: "998e53b2f6b4dbc3fe5a2f46003e09a5"
server_key_version: "encryptionkey@0"
I20260807 20:13:04.233436 30811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "924d4b7598bc412bb671cadbbc4ccee6"
format_stamp: "Formatted at 2026-08-07 20:13:04 on dist-test-slave-2j7r"
server_key: "b33356418eb8a4c57fc9d118c5f8188c"
server_key_iv: "998e53b2f6b4dbc3fe5a2f46003e09a5"
server_key_version: "encryptionkey@0"
I20260807 20:13:04.234652 30811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:13:04.235426 30832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:04.235627 30811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:13:04.235720 30811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "924d4b7598bc412bb671cadbbc4ccee6"
format_stamp: "Formatted at 2026-08-07 20:13:04 on dist-test-slave-2j7r"
server_key: "b33356418eb8a4c57fc9d118c5f8188c"
server_key_iv: "998e53b2f6b4dbc3fe5a2f46003e09a5"
server_key_version: "encryptionkey@0"
I20260807 20:13:04.235776 30811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:04.258495 30811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:04.260687 30811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:04.260808 30811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:04.264623 30811 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:40209
I20260807 20:13:04.264657 30918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:40209 every 8 connection(s)
I20260807 20:13:04.264950 30811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:13:04.265966 30920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:04.268291 30920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6: Bootstrap starting.
I20260807 20:13:04.268879 30920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:04.269187 30920 log.cc:824] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:04.269917 30920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6: No bootstrap required, opened a new log
I20260807 20:13:04.271106 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 30811
I20260807 20:13:04.271206 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:13:04.271374 24355 external_mini_cluster.cc:1428] Setting key 99197c6ba4928eef55e3fb32efd232a6
I20260807 20:13:04.271668 30920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924d4b7598bc412bb671cadbbc4ccee6" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 40209 } }
I20260807 20:13:04.271860 30920 raft_consensus.cc:374] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:04.271932 30920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 924d4b7598bc412bb671cadbbc4ccee6, State: Initialized, Role: FOLLOWER
I20260807 20:13:04.272091 30920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924d4b7598bc412bb671cadbbc4ccee6" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 40209 } }
I20260807 20:13:04.272181 30920 raft_consensus.cc:388] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:13:04.272204 30920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:13:04.272254 30920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:04.272810 30920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924d4b7598bc412bb671cadbbc4ccee6" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 40209 } }
I20260807 20:13:04.272924 30920 leader_election.cc:302] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 924d4b7598bc412bb671cadbbc4ccee6; no voters:
I20260807 20:13:04.273124 30920 leader_election.cc:288] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:13:04.273147 30926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:04.273312 30926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [term 1 LEADER]: Becoming Leader. State: Replica: 924d4b7598bc412bb671cadbbc4ccee6, State: Running, Role: LEADER
I20260807 20:13:04.273411 30920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:04.273490 30926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924d4b7598bc412bb671cadbbc4ccee6" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 40209 } }
I20260807 20:13:04.273840 30926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 924d4b7598bc412bb671cadbbc4ccee6. Latest consensus state: current_term: 1 leader_uuid: "924d4b7598bc412bb671cadbbc4ccee6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924d4b7598bc412bb671cadbbc4ccee6" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 40209 } } }
I20260807 20:13:04.273977 30926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:04.274228 30940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:13:04.274901 30940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:04.274902 30929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "924d4b7598bc412bb671cadbbc4ccee6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "924d4b7598bc412bb671cadbbc4ccee6" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 40209 } } }
I20260807 20:13:04.275125 30929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:04.276680 30940 catalog_manager.cc:1347] Generated new cluster ID: 2a2b6f4ad79a465181e85307b11ace17
I20260807 20:13:04.276733 30940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:13:04 dist-test-slave-2j7r krb5kdc[30794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:04.282932 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:04.271963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45336 (local address 127.23.200.254:40209)
0807 20:13:04.272158 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:04.272169 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:04.272204 (+ 35us) server_negotiation.cc:407] Connection header received
0807 20:13:04.272578 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:04.272585 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:04.272632 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:04.272746 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:04.273550 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.274971 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.275936 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.276455 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.276554 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:04.279294 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:04.279298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:04.279299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:04.279312 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:04.281311 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:04.281706 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:04.281709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:04.281710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:04.281818 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:04.282027 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:04.282028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:04.282028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:04.282310 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:04.282469 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:04.282571 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 20:13:04.282702 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260807 20:13:04.285871 30940 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:13:04.286397 30940 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:04.292491 30940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6: Generated new TSK 0
I20260807 20:13:04.292671 30940 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:13:04.321936 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:40209
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:37955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:04.404311 30960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:04.404525 30960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:04.404549 30960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:04.405706 30960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:04.405746 30960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:04.405771 30960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:13:04.407341 30960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:37955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:40209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:04.407557 30960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:04.407739 30960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:04.412572 30966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:04.412580 30974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:04.412591 30967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:04.412791 30960 server_base.cc:1034] running on GCE node
I20260807 20:13:04.413031 30960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:04.413303 30960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:04.414446 30960 hybrid_clock.cc:648] HybridClock initialized: now 1786133584414434 us; error 28 us; skew 500 ppm
Aug 07 20:13:04 dist-test-slave-2j7r krb5kdc[30794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133584, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:04.416931 30960 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:13:04.417301 30960 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:04.417356 30960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:04.417490 30960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:04.418524 30960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "a710cb523f1f4829bec9ac50eda907f4"
format_stamp: "Formatted at 2026-08-07 20:13:04 on dist-test-slave-2j7r"
server_key: "9575860469367d5da5c1e7b6981f0960"
server_key_iv: "7cd190b1efd5c01cd1d0cbafc409fb2a"
server_key_version: "encryptionkey@0"
I20260807 20:13:04.418851 30960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "a710cb523f1f4829bec9ac50eda907f4"
format_stamp: "Formatted at 2026-08-07 20:13:04 on dist-test-slave-2j7r"
server_key: "9575860469367d5da5c1e7b6981f0960"
server_key_iv: "7cd190b1efd5c01cd1d0cbafc409fb2a"
server_key_version: "encryptionkey@0"
I20260807 20:13:04.420028 30960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:13:04.420642 30981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:04.420794 30960 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:13:04.420877 30960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "a710cb523f1f4829bec9ac50eda907f4"
format_stamp: "Formatted at 2026-08-07 20:13:04 on dist-test-slave-2j7r"
server_key: "9575860469367d5da5c1e7b6981f0960"
server_key_iv: "7cd190b1efd5c01cd1d0cbafc409fb2a"
server_key_version: "encryptionkey@0"
I20260807 20:13:04.420943 30960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:04.439942 30960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:04.441812 30960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:04.441954 30960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:04.442188 30960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:04.442495 30960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:04.442530 30960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:04.442560 30960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:04.442584 30960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:04.448249 30960 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:33153
I20260807 20:13:04.448307 31185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:33153 every 8 connection(s)
I20260807 20:13:04.448568 30960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
Aug 07 20:13:04 dist-test-slave-2j7r krb5kdc[30794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133584, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:04.457264 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 30960
I20260807 20:13:04.457379 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:13:04.457496 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:04.449254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:33169 (local address 127.23.200.254:40209)
0807 20:13:04.449325 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:04.449327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:04.449523 (+ 196us) server_negotiation.cc:407] Connection header received
0807 20:13:04.450030 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:04.450031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:04.450044 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:04.450093 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:04.451053 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.451565 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.452547 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.452758 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.452814 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:04.454943 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:04.454945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:04.454945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:04.454956 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:04.456086 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:04.456468 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:04.456469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:04.456470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:04.456529 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:04.456746 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:04.456747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:04.456747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:04.456861 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:04.456927 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:04.457252 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 20:13:04.457393 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:13:04.457580 24355 external_mini_cluster.cc:1428] Setting key bf5fac2e431c57778febcd9cb235234a
I20260807 20:13:04.457646 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:04.449217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:40209 (local address 127.23.200.193:33169)
0807 20:13:04.449405 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:04.449442 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:04.449933 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:04.450177 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:04.450181 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:04.450457 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:04.450959 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:04.450964 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.451649 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.451650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:04.452465 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:04.452467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.452828 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.452829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:04.452955 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:04.453440 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:04.453449 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:04.454847 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:04.456165 (+ 1318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:04.456172 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:04.456175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:04.456383 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:04.456584 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:04.456585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:04.456585 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:04.456689 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:04.456918 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:04.456925 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:04.457072 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 20:13:04.457214 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 20:13:04.457410 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
I20260807 20:13:04.458292 31186 heartbeater.cc:347] Connected to a master server at 127.23.200.254:40209
I20260807 20:13:04.458405 31186 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:04.458567 31186 heartbeater.cc:511] Master 127.23.200.254:40209 requested a full tablet report, sending...
I20260807 20:13:04.459127 30856 ts_manager.cc:194] Registered new tserver with Master: a710cb523f1f4829bec9ac50eda907f4 (127.23.200.193:33153)
I20260807 20:13:04.459890 30856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:33169
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:13:04.484285 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:40209
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:37955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:04.567835 31192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:04.568063 31192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:04.568089 31192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:04.569375 31192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:04.569422 31192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:04.569459 31192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:13:04.571030 31192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:37955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:40209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:04.571280 31192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:04.571497 31192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:04.576612 31200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:04.576632 31203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:04.576785 31192 server_base.cc:1034] running on GCE node
W20260807 20:13:04.576642 31199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:04.577039 31192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:04.577292 31192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:04.578436 31192 hybrid_clock.cc:648] HybridClock initialized: now 1786133584578416 us; error 38 us; skew 500 ppm
Aug 07 20:13:04 dist-test-slave-2j7r krb5kdc[30794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133584, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:04.581068 31192 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:13:04.581468 31192 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:04.581519 31192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:04.581641 31192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:04.582711 31192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "965aeafb38184fb4bc56073f9bd3eaee"
format_stamp: "Formatted at 2026-08-07 20:13:04 on dist-test-slave-2j7r"
server_key: "5a998cc2d772e9b20252a4d851be4582"
server_key_iv: "7865b411c56aee0c4bdae20589894a7d"
server_key_version: "encryptionkey@0"
I20260807 20:13:04.583057 31192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "965aeafb38184fb4bc56073f9bd3eaee"
format_stamp: "Formatted at 2026-08-07 20:13:04 on dist-test-slave-2j7r"
server_key: "5a998cc2d772e9b20252a4d851be4582"
server_key_iv: "7865b411c56aee0c4bdae20589894a7d"
server_key_version: "encryptionkey@0"
I20260807 20:13:04.584275 31192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:13:04.584939 31212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:04.585129 31192 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:04.585220 31192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "965aeafb38184fb4bc56073f9bd3eaee"
format_stamp: "Formatted at 2026-08-07 20:13:04 on dist-test-slave-2j7r"
server_key: "5a998cc2d772e9b20252a4d851be4582"
server_key_iv: "7865b411c56aee0c4bdae20589894a7d"
server_key_version: "encryptionkey@0"
I20260807 20:13:04.585276 31192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:04.614288 31192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:04.616216 31192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:04.616333 31192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:04.616549 31192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:04.616842 31192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:04.616874 31192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:04.616896 31192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:04.616907 31192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:04.622562 31192 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:35767
I20260807 20:13:04.622634 31385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:35767 every 8 connection(s)
I20260807 20:13:04.622886 31192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
Aug 07 20:13:04 dist-test-slave-2j7r krb5kdc[30794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133584, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:04.630091 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 31192
I20260807 20:13:04.630193 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:13:04.630363 24355 external_mini_cluster.cc:1428] Setting key 70b3a6e8fd58c39828788ef27b946fa8
I20260807 20:13:04.631862 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:04.623617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:43187 (local address 127.23.200.254:40209)
0807 20:13:04.623682 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:04.623684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:04.623876 (+ 192us) server_negotiation.cc:407] Connection header received
0807 20:13:04.624381 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:04.624382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:04.624396 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:04.624448 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:04.625467 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.625983 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.626958 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.627144 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.627193 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:04.629288 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:04.629290 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:04.629290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:04.629297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:04.630510 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:04.630930 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:04.630931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:04.630931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:04.630989 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:04.631240 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:04.631241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:04.631241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:04.631359 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:04.631416 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:04.631684 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 20:13:04.631773 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:13:04.632030 31390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:04.623619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:40209 (local address 127.23.200.194:43187)
0807 20:13:04.623766 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:04.623803 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:04.624263 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:04.624507 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:04.624511 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:04.624825 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:04.625369 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:04.625375 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.626046 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.626046 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:04.626888 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:04.626891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.627207 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.627207 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:04.627307 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:04.627799 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:04.627808 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:04.629193 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:04.630578 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:04.630586 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:04.630589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:04.630864 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:04.631042 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:04.631043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:04.631043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:04.631176 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:04.631421 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:04.631428 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:04.631562 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 20:13:04.631654 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 20:13:04.631808 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260807 20:13:04.632472 31386 heartbeater.cc:347] Connected to a master server at 127.23.200.254:40209
I20260807 20:13:04.632568 31386 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:04.632720 31386 heartbeater.cc:511] Master 127.23.200.254:40209 requested a full tablet report, sending...
I20260807 20:13:04.633155 30856 ts_manager.cc:194] Registered new tserver with Master: 965aeafb38184fb4bc56073f9bd3eaee (127.23.200.194:35767)
I20260807 20:13:04.633525 30856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:43187
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:13:04.656136 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:40209
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:37955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:04.740501 31394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:04.740708 31394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:04.740731 31394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:04.741843 31394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:04.741878 31394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:04.741902 31394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:13:04.743340 31394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:37955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:40209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:04.743522 31394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:04.743700 31394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:04.748565 31405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:04.748677 31404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:04.748562 31394 server_base.cc:1034] running on GCE node
W20260807 20:13:04.748572 31407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:04.748979 31394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:04.749229 31394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:04.750381 31394 hybrid_clock.cc:648] HybridClock initialized: now 1786133584750358 us; error 38 us; skew 500 ppm
Aug 07 20:13:04 dist-test-slave-2j7r krb5kdc[30794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133584, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:04.752806 31394 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:13:04.753144 31394 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:04.753193 31394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:04.753301 31394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:04.754359 31394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "3db83ea9792748528940a3dbb521f01a"
format_stamp: "Formatted at 2026-08-07 20:13:04 on dist-test-slave-2j7r"
server_key: "36c6bce631fec63b5f1cc39bbd5298f2"
server_key_iv: "d65fc6e1af71a638e455f1551453f0ae"
server_key_version: "encryptionkey@0"
I20260807 20:13:04.754694 31394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "3db83ea9792748528940a3dbb521f01a"
format_stamp: "Formatted at 2026-08-07 20:13:04 on dist-test-slave-2j7r"
server_key: "36c6bce631fec63b5f1cc39bbd5298f2"
server_key_iv: "d65fc6e1af71a638e455f1551453f0ae"
server_key_version: "encryptionkey@0"
I20260807 20:13:04.755944 31394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:13:04.756593 31414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:04.756769 31394 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:13:04.756861 31394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "3db83ea9792748528940a3dbb521f01a"
format_stamp: "Formatted at 2026-08-07 20:13:04 on dist-test-slave-2j7r"
server_key: "36c6bce631fec63b5f1cc39bbd5298f2"
server_key_iv: "d65fc6e1af71a638e455f1551453f0ae"
server_key_version: "encryptionkey@0"
I20260807 20:13:04.756919 31394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:04.769527 31394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:04.771524 31394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:04.771636 31394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:04.771845 31394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:04.772128 31394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:04.772161 31394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:04.772182 31394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:04.772197 31394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:04.777958 31394 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:36817
I20260807 20:13:04.778038 31581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:36817 every 8 connection(s)
I20260807 20:13:04.778256 31394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:13:04.781558 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 31394
I20260807 20:13:04.781661 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:13:04.781845 24355 external_mini_cluster.cc:1428] Setting key 1cec96cc1bd4ec117536e9b19778b2d8
Aug 07 20:13:04 dist-test-slave-2j7r krb5kdc[30794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133584, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:04.787190 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:04.778955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:51113 (local address 127.23.200.254:40209)
0807 20:13:04.779015 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:04.779018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:04.779224 (+ 206us) server_negotiation.cc:407] Connection header received
0807 20:13:04.779773 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:04.779774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:04.779788 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:04.779838 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:04.780789 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.781361 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.782516 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.782764 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.782827 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:04.784784 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:04.784786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:04.784786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:04.784793 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:04.786139 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:04.786489 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:04.786490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:04.786490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:04.786536 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:04.786706 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:04.786706 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:04.786706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:04.786800 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:04.786844 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:04.787066 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 20:13:04.787124 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 20:13:04.787371 31588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:04.778936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:40209 (local address 127.23.200.195:51113)
0807 20:13:04.779108 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:04.779144 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:04.779666 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:04.779922 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:04.779926 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:04.780191 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:04.780694 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:04.780699 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.781468 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.781470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:04.782431 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:04.782435 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.782840 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.782842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:04.782962 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:04.783377 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:04.783385 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:04.784702 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:04.786208 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:04.786215 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:04.786217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:04.786432 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:04.786580 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:04.786580 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:04.786580 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:04.786667 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:04.786841 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:04.786844 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:04.786969 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 20:13:04.787046 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 20:13:04.787164 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260807 20:13:04.787794 31582 heartbeater.cc:347] Connected to a master server at 127.23.200.254:40209
I20260807 20:13:04.787909 31582 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:04.788058 31582 heartbeater.cc:511] Master 127.23.200.254:40209 requested a full tablet report, sending...
I20260807 20:13:04.788379 30856 ts_manager.cc:194] Registered new tserver with Master: 3db83ea9792748528940a3dbb521f01a (127.23.200.195:36817)
I20260807 20:13:04.788705 30856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:51113
I20260807 20:13:04.793550 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:13:04.800022 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:04.794734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45342 (local address 127.23.200.254:40209)
0807 20:13:04.794791 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:04.794793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:04.794884 (+ 91us) server_negotiation.cc:407] Connection header received
0807 20:13:04.794927 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:04.794928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:04.794937 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:04.794980 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:04.795498 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.795962 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.796689 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.796915 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.796971 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:04.798028 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:04.798030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:04.798030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:04.798037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:04.798984 (+ 947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:04.799277 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:04.799278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:04.799279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:04.799336 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:04.799496 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:04.799497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:04.799498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:04.799648 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:04.799735 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:04.799854 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 20:13:04.799933 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 20:13:04.801442 30856 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:13:04.803013 30856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:13:04.809552 31602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:04.805258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:35767 (local address 127.0.0.1:45900)
0807 20:13:04.805795 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:04.805806 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:04.805862 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:04.806115 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:04.806116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:04.806117 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:04.806155 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:04.806156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.807464 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.807465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:04.808579 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:04.808581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.809260 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.809261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:04.809341 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:04.809357 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 20:13:04.809423 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 20:13:04.809475 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":27,"thread_start_us":101,"threads_started":1}
I20260807 20:13:04.809652 31604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:04.805437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47426 (local address 127.23.200.193:33153)
0807 20:13:04.805826 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:04.805830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:04.805841 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:13:04.805882 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:04.805884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:04.805913 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:04.806005 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:04.806204 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.807348 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.808453 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.809194 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.809265 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:04.809320 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:04.809531 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 20:13:04.809577 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":76,"threads_started":1}
I20260807 20:13:04.809792 31609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:04.805632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45900 (local address 127.23.200.194:35767)
0807 20:13:04.805863 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:04.805865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:04.805874 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:04.805914 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:04.805915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:04.805928 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:04.806030 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:04.806227 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.807346 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.808639 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.809179 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.809242 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:04.809303 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:04.809678 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 20:13:04.809720 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":81,"threads_started":1}
I20260807 20:13:04.809907 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:04.805618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58320 (local address 127.23.200.195:36817)
0807 20:13:04.805791 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:04.805795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:04.806224 (+ 429us) server_negotiation.cc:407] Connection header received
0807 20:13:04.806267 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:04.806270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:04.806284 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:04.806352 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:04.806476 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.807136 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.808594 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.809327 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.809388 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:04.809438 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:04.809771 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 20:13:04.809821 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260807 20:13:04.809953 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:04.805250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:33153 (local address 127.0.0.1:47426)
0807 20:13:04.805413 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:04.805455 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:04.805515 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:04.806027 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:04.806031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:04.806033 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:04.806106 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:04.806108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.807465 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.807466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:04.808374 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:04.808377 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.809358 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.809359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:04.809445 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:04.809449 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:13:04.809497 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 20:13:04.809537 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":34,"thread_start_us":68,"threads_started":1}
I20260807 20:13:04.810036 31606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:04.805508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:36817 (local address 127.0.0.1:58320)
0807 20:13:04.806115 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:04.806124 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:04.806171 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:04.806361 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:04.806362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:04.806364 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:04.806418 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:04.806420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.807230 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.807231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:04.808512 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:04.808515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:04.809468 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:04.809468 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:04.809520 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:04.809523 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:13:04.809564 (+ 41us) client_negotiation.cc:240] Negotiation successful
0807 20:13:04.809597 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":37,"thread_start_us":83,"threads_started":1}
I20260807 20:13:04.810524 31290 tablet_service.cc:1467] Processing CreateTablet for tablet 33aafab795bc4f028656b5320a1cb4f3 (DEFAULT_TABLE table=test-table [id=54e4cf11508645b79693ff3f3d3effa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:04.810633 31483 tablet_service.cc:1467] Processing CreateTablet for tablet 33aafab795bc4f028656b5320a1cb4f3 (DEFAULT_TABLE table=test-table [id=54e4cf11508645b79693ff3f3d3effa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:04.810673 31066 tablet_service.cc:1467] Processing CreateTablet for tablet 33aafab795bc4f028656b5320a1cb4f3 (DEFAULT_TABLE table=test-table [id=54e4cf11508645b79693ff3f3d3effa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:04.810930 31290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33aafab795bc4f028656b5320a1cb4f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:04.810951 31066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33aafab795bc4f028656b5320a1cb4f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:04.810954 31483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33aafab795bc4f028656b5320a1cb4f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:04.813443 31610 tablet_bootstrap.cc:492] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a: Bootstrap starting.
I20260807 20:13:04.814059 31610 tablet_bootstrap.cc:654] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:04.814157 31613 tablet_bootstrap.cc:492] T 33aafab795bc4f028656b5320a1cb4f3 P a710cb523f1f4829bec9ac50eda907f4: Bootstrap starting.
I20260807 20:13:04.814791 31613 tablet_bootstrap.cc:654] T 33aafab795bc4f028656b5320a1cb4f3 P a710cb523f1f4829bec9ac50eda907f4: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:04.814854 31610 log.cc:824] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:04.815383 31613 log.cc:824] T 33aafab795bc4f028656b5320a1cb4f3 P a710cb523f1f4829bec9ac50eda907f4: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:04.815644 31610 tablet_bootstrap.cc:492] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a: No bootstrap required, opened a new log
I20260807 20:13:04.815717 31610 ts_tablet_manager.cc:1397] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:13:04.815733 31611 tablet_bootstrap.cc:492] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee: Bootstrap starting.
I20260807 20:13:04.816074 31613 tablet_bootstrap.cc:492] T 33aafab795bc4f028656b5320a1cb4f3 P a710cb523f1f4829bec9ac50eda907f4: No bootstrap required, opened a new log
I20260807 20:13:04.816133 31613 ts_tablet_manager.cc:1397] T 33aafab795bc4f028656b5320a1cb4f3 P a710cb523f1f4829bec9ac50eda907f4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:13:04.816349 31611 tablet_bootstrap.cc:654] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:04.816919 31611 log.cc:824] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:04.817628 31610 raft_consensus.cc:348] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a710cb523f1f4829bec9ac50eda907f4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 33153 } } peers { permanent_uuid: "965aeafb38184fb4bc56073f9bd3eaee" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35767 } } peers { permanent_uuid: "3db83ea9792748528940a3dbb521f01a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36817 } }
I20260807 20:13:04.817759 31610 raft_consensus.cc:374] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:04.817780 31610 raft_consensus.cc:729] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3db83ea9792748528940a3dbb521f01a, State: Initialized, Role: FOLLOWER
I20260807 20:13:04.817736 31613 raft_consensus.cc:348] T 33aafab795bc4f028656b5320a1cb4f3 P a710cb523f1f4829bec9ac50eda907f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a710cb523f1f4829bec9ac50eda907f4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 33153 } } peers { permanent_uuid: "965aeafb38184fb4bc56073f9bd3eaee" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35767 } } peers { permanent_uuid: "3db83ea9792748528940a3dbb521f01a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36817 } }
I20260807 20:13:04.817865 31610 consensus_queue.cc:260] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a710cb523f1f4829bec9ac50eda907f4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 33153 } } peers { permanent_uuid: "965aeafb38184fb4bc56073f9bd3eaee" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35767 } } peers { permanent_uuid: "3db83ea9792748528940a3dbb521f01a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36817 } }
I20260807 20:13:04.817915 31613 raft_consensus.cc:374] T 33aafab795bc4f028656b5320a1cb4f3 P a710cb523f1f4829bec9ac50eda907f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:04.817968 31613 raft_consensus.cc:729] T 33aafab795bc4f028656b5320a1cb4f3 P a710cb523f1f4829bec9ac50eda907f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a710cb523f1f4829bec9ac50eda907f4, State: Initialized, Role: FOLLOWER
I20260807 20:13:04.818066 31613 consensus_queue.cc:260] T 33aafab795bc4f028656b5320a1cb4f3 P a710cb523f1f4829bec9ac50eda907f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a710cb523f1f4829bec9ac50eda907f4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 33153 } } peers { permanent_uuid: "965aeafb38184fb4bc56073f9bd3eaee" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35767 } } peers { permanent_uuid: "3db83ea9792748528940a3dbb521f01a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36817 } }
I20260807 20:13:04.818109 31610 ts_tablet_manager.cc:1428] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 20:13:04.818217 31582 heartbeater.cc:503] Master 127.23.200.254:40209 was elected leader, sending a full tablet report...
I20260807 20:13:04.818306 31186 heartbeater.cc:503] Master 127.23.200.254:40209 was elected leader, sending a full tablet report...
I20260807 20:13:04.818569 31613 ts_tablet_manager.cc:1428] T 33aafab795bc4f028656b5320a1cb4f3 P a710cb523f1f4829bec9ac50eda907f4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:13:04.818744 31611 tablet_bootstrap.cc:492] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee: No bootstrap required, opened a new log
I20260807 20:13:04.818800 31611 ts_tablet_manager.cc:1397] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 20:13:04.820307 31611 raft_consensus.cc:348] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a710cb523f1f4829bec9ac50eda907f4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 33153 } } peers { permanent_uuid: "965aeafb38184fb4bc56073f9bd3eaee" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35767 } } peers { permanent_uuid: "3db83ea9792748528940a3dbb521f01a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36817 } }
I20260807 20:13:04.820438 31611 raft_consensus.cc:374] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:04.820470 31611 raft_consensus.cc:729] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 965aeafb38184fb4bc56073f9bd3eaee, State: Initialized, Role: FOLLOWER
I20260807 20:13:04.820580 31611 consensus_queue.cc:260] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a710cb523f1f4829bec9ac50eda907f4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 33153 } } peers { permanent_uuid: "965aeafb38184fb4bc56073f9bd3eaee" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35767 } } peers { permanent_uuid: "3db83ea9792748528940a3dbb521f01a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36817 } }
I20260807 20:13:04.820852 31611 ts_tablet_manager.cc:1428] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:13:04.820887 31386 heartbeater.cc:503] Master 127.23.200.254:40209 was elected leader, sending a full tablet report...
W20260807 20:13:04.873411 31387 tablet.cc:2369] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:13:04.949249 31187 tablet.cc:2369] T 33aafab795bc4f028656b5320a1cb4f3 P a710cb523f1f4829bec9ac50eda907f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:13:05.028781 31584 tablet.cc:2369] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:13:05.057495 31622 raft_consensus.cc:482] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:13:05.057657 31622 raft_consensus.cc:504] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a710cb523f1f4829bec9ac50eda907f4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 33153 } } peers { permanent_uuid: "965aeafb38184fb4bc56073f9bd3eaee" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35767 } } peers { permanent_uuid: "3db83ea9792748528940a3dbb521f01a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36817 } }
I20260807 20:13:05.058043 31622 leader_election.cc:288] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a710cb523f1f4829bec9ac50eda907f4 (127.23.200.193:33153), 3db83ea9792748528940a3dbb521f01a (127.23.200.195:36817)
I20260807 20:13:05.061106 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.058416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:49463 (local address 127.23.200.195:36817)
0807 20:13:05.058475 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:05.058477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:05.058603 (+ 126us) server_negotiation.cc:407] Connection header received
0807 20:13:05.058652 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:05.058653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:05.058664 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:05.058713 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:05.058873 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:05.059337 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:05.060329 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:05.060723 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:05.060787 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:05.060796 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:05.060979 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 20:13:05.061024 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 20:13:05.061060 31390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.058234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:33153 (local address 127.23.200.194:40315)
0807 20:13:05.058288 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:05.058300 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:05.058348 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:05.058569 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:05.058570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:05.058572 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:05.058632 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:05.058634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:05.059313 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:05.059314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:05.060213 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:05.060215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:05.060768 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:05.060769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:05.060839 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:05.060843 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:13:05.060919 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 20:13:05.060955 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 20:13:05.061123 31604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.058326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:40315 (local address 127.23.200.193:33153)
0807 20:13:05.058382 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:05.058384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:05.058395 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:13:05.058453 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:05.058455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:05.058464 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:05.058542 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:05.058702 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:05.059242 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:05.060292 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:05.060688 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:05.060768 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:05.060777 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:05.060948 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 20:13:05.060988 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:13:05.061177 31625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.058336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:36817 (local address 127.23.200.194:49463)
0807 20:13:05.058529 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:05.058539 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:05.058597 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:05.058724 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:05.058725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:05.058726 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:05.058797 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:05.058799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:05.059442 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:05.059443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:05.060241 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:05.060244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:05.060825 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:05.060826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:05.060900 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:05.060905 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:13:05.060961 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 20:13:05.060997 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":108,"threads_started":1}
I20260807 20:13:05.061458 31107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33aafab795bc4f028656b5320a1cb4f3" candidate_uuid: "965aeafb38184fb4bc56073f9bd3eaee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a710cb523f1f4829bec9ac50eda907f4" is_pre_election: true
I20260807 20:13:05.061504 31512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33aafab795bc4f028656b5320a1cb4f3" candidate_uuid: "965aeafb38184fb4bc56073f9bd3eaee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3db83ea9792748528940a3dbb521f01a" is_pre_election: true
I20260807 20:13:05.061614 31107 raft_consensus.cc:2454] T 33aafab795bc4f028656b5320a1cb4f3 P a710cb523f1f4829bec9ac50eda907f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965aeafb38184fb4bc56073f9bd3eaee in term 0.
I20260807 20:13:05.061643 31512 raft_consensus.cc:2454] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965aeafb38184fb4bc56073f9bd3eaee in term 0.
I20260807 20:13:05.061863 31214 leader_election.cc:302] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3db83ea9792748528940a3dbb521f01a, 965aeafb38184fb4bc56073f9bd3eaee; no voters:
I20260807 20:13:05.062006 31622 raft_consensus.cc:2781] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:13:05.062068 31622 raft_consensus.cc:482] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:13:05.062083 31622 raft_consensus.cc:3037] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:05.062755 31622 raft_consensus.cc:504] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a710cb523f1f4829bec9ac50eda907f4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 33153 } } peers { permanent_uuid: "965aeafb38184fb4bc56073f9bd3eaee" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35767 } } peers { permanent_uuid: "3db83ea9792748528940a3dbb521f01a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36817 } }
I20260807 20:13:05.062870 31622 leader_election.cc:288] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [CANDIDATE]: Term 1 election: Requested vote from peers a710cb523f1f4829bec9ac50eda907f4 (127.23.200.193:33153), 3db83ea9792748528940a3dbb521f01a (127.23.200.195:36817)
I20260807 20:13:05.063057 31107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33aafab795bc4f028656b5320a1cb4f3" candidate_uuid: "965aeafb38184fb4bc56073f9bd3eaee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a710cb523f1f4829bec9ac50eda907f4"
I20260807 20:13:05.063092 31512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33aafab795bc4f028656b5320a1cb4f3" candidate_uuid: "965aeafb38184fb4bc56073f9bd3eaee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3db83ea9792748528940a3dbb521f01a"
I20260807 20:13:05.063124 31107 raft_consensus.cc:3037] T 33aafab795bc4f028656b5320a1cb4f3 P a710cb523f1f4829bec9ac50eda907f4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:05.063181 31512 raft_consensus.cc:3037] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:05.063800 31107 raft_consensus.cc:2454] T 33aafab795bc4f028656b5320a1cb4f3 P a710cb523f1f4829bec9ac50eda907f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965aeafb38184fb4bc56073f9bd3eaee in term 1.
I20260807 20:13:05.063934 31512 raft_consensus.cc:2454] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965aeafb38184fb4bc56073f9bd3eaee in term 1.
I20260807 20:13:05.063989 31214 leader_election.cc:302] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965aeafb38184fb4bc56073f9bd3eaee, a710cb523f1f4829bec9ac50eda907f4; no voters:
I20260807 20:13:05.064067 31622 raft_consensus.cc:2781] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:05.064239 31622 raft_consensus.cc:686] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [term 1 LEADER]: Becoming Leader. State: Replica: 965aeafb38184fb4bc56073f9bd3eaee, State: Running, Role: LEADER
I20260807 20:13:05.064337 31622 consensus_queue.cc:237] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a710cb523f1f4829bec9ac50eda907f4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 33153 } } peers { permanent_uuid: "965aeafb38184fb4bc56073f9bd3eaee" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35767 } } peers { permanent_uuid: "3db83ea9792748528940a3dbb521f01a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36817 } }
I20260807 20:13:05.065235 30856 catalog_manager.cc:5495] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee reported cstate change: term changed from 0 to 1, leader changed from <none> to 965aeafb38184fb4bc56073f9bd3eaee (127.23.200.194). New cstate: current_term: 1 leader_uuid: "965aeafb38184fb4bc56073f9bd3eaee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a710cb523f1f4829bec9ac50eda907f4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 33153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965aeafb38184fb4bc56073f9bd3eaee" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3db83ea9792748528940a3dbb521f01a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36817 } health_report { overall_health: UNKNOWN } } }
I20260807 20:13:05.095410 31609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.092320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45912 (local address 127.23.200.194:35767)
0807 20:13:05.092399 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:05.092401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:05.092433 (+ 32us) server_negotiation.cc:407] Connection header received
0807 20:13:05.092477 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:05.092478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:05.092502 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:05.092576 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:13:05.092779 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:05.093551 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:05.094456 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:05.094730 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:05.094790 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:05.095000 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:13:05.095164 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:13:05.095247 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:05.095270 (+ 23us) server_negotiation.cc:299] Negotiation successful
0807 20:13:05.095295 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 20:13:05.097707 31512 raft_consensus.cc:1261] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a [term 1 FOLLOWER]: Refusing update from remote peer 965aeafb38184fb4bc56073f9bd3eaee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:05.097756 31107 raft_consensus.cc:1261] T 33aafab795bc4f028656b5320a1cb4f3 P a710cb523f1f4829bec9ac50eda907f4 [term 1 FOLLOWER]: Refusing update from remote peer 965aeafb38184fb4bc56073f9bd3eaee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:05.098116 31622 consensus_queue.cc:1035] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [LEADER]: Connected to new peer: Peer: permanent_uuid: "a710cb523f1f4829bec9ac50eda907f4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 33153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:05.098300 31622 consensus_queue.cc:1035] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3db83ea9792748528940a3dbb521f01a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 36817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:05.104892 30857 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:05.105005 30857 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:05.105770 30857 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 924d4b7598bc412bb671cadbbc4ccee6: Sending DeleteTablet for 3 replicas of tablet 33aafab795bc4f028656b5320a1cb4f3
I20260807 20:13:05.106102 31290 tablet_service.cc:1514] Processing DeleteTablet for tablet 33aafab795bc4f028656b5320a1cb4f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:05 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:45900
I20260807 20:13:05.106172 31483 tablet_service.cc:1514] Processing DeleteTablet for tablet 33aafab795bc4f028656b5320a1cb4f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:05 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:58320
I20260807 20:13:05.106179 31066 tablet_service.cc:1514] Processing DeleteTablet for tablet 33aafab795bc4f028656b5320a1cb4f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:05 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:47426
I20260807 20:13:05.106297 31640 tablet_replica.cc:330] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee: stopping tablet replica
I20260807 20:13:05.106379 31640 raft_consensus.cc:2229] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:13:05.106485 31640 raft_consensus.cc:2258] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:05.106575 31645 tablet_replica.cc:330] T 33aafab795bc4f028656b5320a1cb4f3 P a710cb523f1f4829bec9ac50eda907f4: stopping tablet replica
I20260807 20:13:05.106675 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 30960
I20260807 20:13:05.112839 31644 tablet_replica.cc:330] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a: stopping tablet replica
I20260807 20:13:05.112987 31644 raft_consensus.cc:2229] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:05.113175 31644 raft_consensus.cc:2258] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:05.126407 31644 ts_tablet_manager.cc:1905] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:05.128351 31644 ts_tablet_manager.cc:1918] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:13:05.128451 31644 log.cc:1196] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/33aafab795bc4f028656b5320a1cb4f3
I20260807 20:13:05.128743 31644 ts_tablet_manager.cc:1939] T 33aafab795bc4f028656b5320a1cb4f3 P 3db83ea9792748528940a3dbb521f01a: Deleting consensus metadata
I20260807 20:13:05.129341 30835 catalog_manager.cc:4841] TS 3db83ea9792748528940a3dbb521f01a (127.23.200.195:36817): tablet 33aafab795bc4f028656b5320a1cb4f3 (table test-table [id=54e4cf11508645b79693ff3f3d3effa4]) successfully deleted
I20260807 20:13:05.130232 31640 ts_tablet_manager.cc:1905] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 20:13:05.130939 30835 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:33153 (error 108)
W20260807 20:13:05.131646 31603 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:05.131349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:33153 (local address 127.0.0.1:47436)
0807 20:13:05.131400 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:05.131412 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:05.131464 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:05.131597 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:33153: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 20:13:05.131757 30835 catalog_manager.cc:4584] TS a710cb523f1f4829bec9ac50eda907f4 (127.23.200.193:33153): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 33aafab795bc4f028656b5320a1cb4f3: Network error: Client connection negotiation failed: client connection to 127.23.200.193:33153: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 20:13:05.131876 31640 ts_tablet_manager.cc:1918] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:13:05.131949 31640 log.cc:1196] T 33aafab795bc4f028656b5320a1cb4f3 P 965aeafb38184fb4bc56073f9bd3eaee: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/33aafab795bc4f028656b5320a1cb4f3
I20260807 20:13:05.132045 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 31192
I20260807 20:13:05.143000 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 31394
I20260807 20:13:05.143112 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.142968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:35767 (local address 127.0.0.1:45916)
0807 20:13:05.143025 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:05.143072 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.194:35767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 20:13:05.155351 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 30811
2026-08-07T20:13:05Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2310 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T20:13:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:05Z Disabled control of system clock
I20260807 20:13:05.189801 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:37125
--webserver_interface=127.23.200.254
--webserver_port=0
--builtin_ntp_servers=127.23.200.212:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:37125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 20:13:05.279066 31653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:05.279278 31653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:05.279307 31653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:05.280421 31653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:05.280462 31653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:05.280479 31653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:05.280494 31653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:05.280509 31653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:13:05.281893 31653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:37125
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:37125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:05.282143 31653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:05.282433 31653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:05.287111 31658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:05.287148 31661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:05.287267 31653 server_base.cc:1034] running on GCE node
W20260807 20:13:05.287377 31663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:05.287653 31653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:05.288054 31653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:05.289240 31653 hybrid_clock.cc:648] HybridClock initialized: now 1786133585289220 us; error 34 us; skew 500 ppm
I20260807 20:13:05.290759 31653 webserver.cc:460] Webserver started at http://127.23.200.254:40463/ using document root <none> and password file <none>
I20260807 20:13:05.291010 31653 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:05.291066 31653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:05.291199 31653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:05.292660 31653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "7c24c789eaf0487887e4906d38598397"
format_stamp: "Formatted at 2026-08-07 20:13:05 on dist-test-slave-2j7r"
server_key: "c62f48be9977e1663b9ea0f9143fc37f"
server_key_iv: "bf74bbfa5002800ec9b2ea4945cf5690"
server_key_version: "encryptionkey@0"
I20260807 20:13:05.293126 31653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "7c24c789eaf0487887e4906d38598397"
format_stamp: "Formatted at 2026-08-07 20:13:05 on dist-test-slave-2j7r"
server_key: "c62f48be9977e1663b9ea0f9143fc37f"
server_key_iv: "bf74bbfa5002800ec9b2ea4945cf5690"
server_key_version: "encryptionkey@0"
I20260807 20:13:05.294803 31653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:13:05.295694 31670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:05.295866 31653 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:05.295974 31653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "7c24c789eaf0487887e4906d38598397"
format_stamp: "Formatted at 2026-08-07 20:13:05 on dist-test-slave-2j7r"
server_key: "c62f48be9977e1663b9ea0f9143fc37f"
server_key_iv: "bf74bbfa5002800ec9b2ea4945cf5690"
server_key_version: "encryptionkey@0"
I20260807 20:13:05.296051 31653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:05.301020 31653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:05.301355 31653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:05.301493 31653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:05.306406 31653 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:37125
I20260807 20:13:05.306439 31751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:37125 every 8 connection(s)
I20260807 20:13:05.306838 31653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:13:05.307557 31752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:05.310331 31752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397: Bootstrap starting.
I20260807 20:13:05.310904 31752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:05.311173 31752 log.cc:824] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:05.311861 31752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397: No bootstrap required, opened a new log
I20260807 20:13:05.313510 31752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c24c789eaf0487887e4906d38598397" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37125 } }
I20260807 20:13:05.313650 31752 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:05.313673 31752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c24c789eaf0487887e4906d38598397, State: Initialized, Role: FOLLOWER
I20260807 20:13:05.313812 31752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c24c789eaf0487887e4906d38598397" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37125 } }
I20260807 20:13:05.313876 31752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:13:05.313918 31752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:13:05.313964 31752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:05.314332 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 31653
I20260807 20:13:05.314451 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:13:05.314538 31752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c24c789eaf0487887e4906d38598397" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37125 } }
I20260807 20:13:05.314640 24355 external_mini_cluster.cc:1428] Setting key ec056294b35dcb4c11b48ad33e15e955
I20260807 20:13:05.314637 31752 leader_election.cc:302] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c24c789eaf0487887e4906d38598397; no voters:
I20260807 20:13:05.314803 31752 leader_election.cc:288] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:13:05.314847 31758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:05.315040 31758 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [term 1 LEADER]: Becoming Leader. State: Replica: 7c24c789eaf0487887e4906d38598397, State: Running, Role: LEADER
I20260807 20:13:05.315084 31752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:05.315137 31758 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c24c789eaf0487887e4906d38598397" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37125 } }
I20260807 20:13:05.315500 31758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c24c789eaf0487887e4906d38598397. Latest consensus state: current_term: 1 leader_uuid: "7c24c789eaf0487887e4906d38598397" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c24c789eaf0487887e4906d38598397" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37125 } } }
I20260807 20:13:05.315579 31758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:05.315809 31759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c24c789eaf0487887e4906d38598397" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c24c789eaf0487887e4906d38598397" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37125 } } }
I20260807 20:13:05.315948 31759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397 [sys.catalog]: This master's current role is: LEADER
W20260807 20:13:05.316972 31792 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:13:05.317026 31792 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:13:05.317188 31779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:13:05.317341 31779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:05.318986 31779 catalog_manager.cc:1347] Generated new cluster ID: 03bdd1f595204e0e84ec606e65b36bd2
I20260807 20:13:05.319069 31779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:13:05.319204 31762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.315231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53862 (local address 127.23.200.254:37125)
0807 20:13:05.315405 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:05.315415 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:05.316046 (+ 631us) server_negotiation.cc:407] Connection header received
0807 20:13:05.316178 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:05.316185 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:05.316230 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:05.316472 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:05.318412 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:05.318416 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:05.318416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:05.318421 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:05.318468 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:05.318484 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:05.318791 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:05.318822 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 20:13:05.318881 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":66,"threads_started":1}
I20260807 20:13:05.332046 31779 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:13:05.332226 31779 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:05.337635 31779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397: Generated new TSK 0
I20260807 20:13:05.337839 31779 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:13:05.343189 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37125
--builtin_ntp_servers=127.23.200.212:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 20:13:05.463203 31799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:05.463492 31799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:05.463531 31799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:05.465246 31799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:05.465304 31799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:05.465344 31799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:13:05.467386 31799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:05.467697 31799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:05.468089 31799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:05.473469 31806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:05.473443 31808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:05.473713 31799 server_base.cc:1034] running on GCE node
W20260807 20:13:05.473445 31810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:05.473965 31799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:05.474241 31799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:05.475410 31799 hybrid_clock.cc:648] HybridClock initialized: now 1786133585475391 us; error 33 us; skew 500 ppm
I20260807 20:13:05.486006 31799 webserver.cc:460] Webserver started at http://127.23.200.193:38361/ using document root <none> and password file <none>
I20260807 20:13:05.486323 31799 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:05.486390 31799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:05.486526 31799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:05.488036 31799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d"
format_stamp: "Formatted at 2026-08-07 20:13:05 on dist-test-slave-2j7r"
server_key: "2abfd45609a0968398ff389105a0b51a"
server_key_iv: "48c83cbd1f218c2d1533783c912824ca"
server_key_version: "encryptionkey@0"
I20260807 20:13:05.488363 31799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d"
format_stamp: "Formatted at 2026-08-07 20:13:05 on dist-test-slave-2j7r"
server_key: "2abfd45609a0968398ff389105a0b51a"
server_key_iv: "48c83cbd1f218c2d1533783c912824ca"
server_key_version: "encryptionkey@0"
I20260807 20:13:05.489701 31799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:13:05.490537 31820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:05.490689 31799 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:05.490783 31799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d"
format_stamp: "Formatted at 2026-08-07 20:13:05 on dist-test-slave-2j7r"
server_key: "2abfd45609a0968398ff389105a0b51a"
server_key_iv: "48c83cbd1f218c2d1533783c912824ca"
server_key_version: "encryptionkey@0"
I20260807 20:13:05.490844 31799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:05.494251 31799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:05.494490 31799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:05.494581 31799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:05.494737 31799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:05.495113 31799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:05.495149 31799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:05.495169 31799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:05.495191 31799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:05.500799 31799 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:37673
I20260807 20:13:05.500836 32016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:37673 every 8 connection(s)
I20260807 20:13:05.501190 31799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:13:05.504091 31762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.501890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:58449 (local address 127.23.200.254:37125)
0807 20:13:05.501975 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:05.501977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:05.502164 (+ 187us) server_negotiation.cc:407] Connection header received
0807 20:13:05.502664 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:05.502665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:05.502674 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:05.502733 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:05.503828 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:05.503830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:05.503830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:05.503834 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:05.503852 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:05.503857 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:05.503905 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:05.504001 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 20:13:05.504017 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 20:13:05.504256 32019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.501842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37125 (local address 127.23.200.193:58449)
0807 20:13:05.502047 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:05.502081 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:05.502559 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:05.502808 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:05.502811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:05.502819 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:05.503658 (+ 839us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:13:05.503659 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:05.503687 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:13:05.503717 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:05.503929 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:05.503934 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:05.503937 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:13:05.504003 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 20:13:05.504066 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":83,"threads_started":1}
I20260807 20:13:05.504719 32017 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37125
I20260807 20:13:05.504813 32017 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:05.504966 32017 heartbeater.cc:511] Master 127.23.200.254:37125 requested a full tablet report, sending...
I20260807 20:13:05.505489 31699 ts_manager.cc:194] Registered new tserver with Master: 9cd53247f4624ea7bb6a4c6aba9bef3d (127.23.200.193:37673)
I20260807 20:13:05.506276 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 31799
I20260807 20:13:05.506358 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:13:05.506538 24355 external_mini_cluster.cc:1428] Setting key 0095fe7c238abca9b2d512bb2f8a9f30
I20260807 20:13:05.507311 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37125
--builtin_ntp_servers=127.23.200.212:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 20:13:05.589630 32020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:05.589841 32020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:05.589864 32020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:05.590993 32020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:05.591030 32020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:05.591055 32020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:13:05.592350 32020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:05.592559 32020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:05.592819 32020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:13:05.597849 32020 server_base.cc:1034] running on GCE node
W20260807 20:13:05.597819 32033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:05.597826 32028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:05.597805 32026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:05.598300 32020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:05.598548 32020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:05.599686 32020 hybrid_clock.cc:648] HybridClock initialized: now 1786133585599670 us; error 31 us; skew 500 ppm
I20260807 20:13:05.600896 32020 webserver.cc:460] Webserver started at http://127.23.200.194:43313/ using document root <none> and password file <none>
I20260807 20:13:05.601145 32020 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:05.601199 32020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:05.601337 32020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:05.602443 32020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "e21c43913f5f4e87a0039c3fed88ef62"
format_stamp: "Formatted at 2026-08-07 20:13:05 on dist-test-slave-2j7r"
server_key: "f89970cf190f0c22b9e09c303af5f17b"
server_key_iv: "f17024bb169da3bb103e5f90a7271137"
server_key_version: "encryptionkey@0"
I20260807 20:13:05.602767 32020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "e21c43913f5f4e87a0039c3fed88ef62"
format_stamp: "Formatted at 2026-08-07 20:13:05 on dist-test-slave-2j7r"
server_key: "f89970cf190f0c22b9e09c303af5f17b"
server_key_iv: "f17024bb169da3bb103e5f90a7271137"
server_key_version: "encryptionkey@0"
I20260807 20:13:05.603992 32020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:13:05.604655 32043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:05.604791 32020 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:05.604873 32020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "e21c43913f5f4e87a0039c3fed88ef62"
format_stamp: "Formatted at 2026-08-07 20:13:05 on dist-test-slave-2j7r"
server_key: "f89970cf190f0c22b9e09c303af5f17b"
server_key_iv: "f17024bb169da3bb103e5f90a7271137"
server_key_version: "encryptionkey@0"
I20260807 20:13:05.604936 32020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:05.608135 32020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:05.608372 32020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:05.608500 32020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:05.608683 32020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:05.608971 32020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:05.609004 32020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:05.609030 32020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:05.609046 32020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:05.614509 32020 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:37693
I20260807 20:13:05.614547 32236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:37693 every 8 connection(s)
I20260807 20:13:05.614837 32020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:13:05.617750 31762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.615563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:37581 (local address 127.23.200.254:37125)
0807 20:13:05.615657 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:05.615678 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:05.615843 (+ 165us) server_negotiation.cc:407] Connection header received
0807 20:13:05.616364 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:05.616366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:05.616374 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:05.616429 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:05.617511 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:05.617512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:05.617513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:05.617518 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:05.617535 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:05.617540 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:05.617590 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:05.617665 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 20:13:05.617680 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 20:13:05.617918 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.615545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37125 (local address 127.23.200.194:37581)
0807 20:13:05.615726 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:05.615767 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:05.616268 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:05.616505 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:05.616509 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:05.616517 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:05.617352 (+ 835us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:13:05.617353 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:05.617381 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:13:05.617426 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:05.617595 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:05.617600 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:05.617603 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:13:05.617671 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 20:13:05.617731 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
I20260807 20:13:05.618403 32240 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37125
I20260807 20:13:05.618496 32240 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:05.618642 32240 heartbeater.cc:511] Master 127.23.200.254:37125 requested a full tablet report, sending...
I20260807 20:13:05.618992 31699 ts_manager.cc:194] Registered new tserver with Master: e21c43913f5f4e87a0039c3fed88ef62 (127.23.200.194:37693)
I20260807 20:13:05.622555 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 32020
I20260807 20:13:05.622633 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:13:05.622771 24355 external_mini_cluster.cc:1428] Setting key d2b35ae53325260893cab61a10dfdb51
I20260807 20:13:05.623564 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37125
--builtin_ntp_servers=127.23.200.212:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 20:13:05.705776 32246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:05.705996 32246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:05.706019 32246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:05.707123 32246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:05.707157 32246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:05.707181 32246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:13:05.708562 32246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:05.708750 32246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:05.708997 32246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:05.713831 32256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:05.713871 32253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:05.713836 32252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:05.714037 32246 server_base.cc:1034] running on GCE node
I20260807 20:13:05.714224 32246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:05.714437 32246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:05.715562 32246 hybrid_clock.cc:648] HybridClock initialized: now 1786133585715548 us; error 29 us; skew 500 ppm
I20260807 20:13:05.716811 32246 webserver.cc:460] Webserver started at http://127.23.200.195:39123/ using document root <none> and password file <none>
I20260807 20:13:05.717026 32246 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:05.717072 32246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:05.717204 32246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:05.718218 32246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "96f55e1632eb4aa6948056ec0c9af772"
format_stamp: "Formatted at 2026-08-07 20:13:05 on dist-test-slave-2j7r"
server_key: "d5546a31f9c7caf24028530fdaa19e9f"
server_key_iv: "f6b52fe61849f946441bcfcb60497adc"
server_key_version: "encryptionkey@0"
I20260807 20:13:05.718529 32246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "96f55e1632eb4aa6948056ec0c9af772"
format_stamp: "Formatted at 2026-08-07 20:13:05 on dist-test-slave-2j7r"
server_key: "d5546a31f9c7caf24028530fdaa19e9f"
server_key_iv: "f6b52fe61849f946441bcfcb60497adc"
server_key_version: "encryptionkey@0"
I20260807 20:13:05.719761 32246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:13:05.720449 32270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:05.720629 32246 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:05.720713 32246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "96f55e1632eb4aa6948056ec0c9af772"
format_stamp: "Formatted at 2026-08-07 20:13:05 on dist-test-slave-2j7r"
server_key: "d5546a31f9c7caf24028530fdaa19e9f"
server_key_iv: "f6b52fe61849f946441bcfcb60497adc"
server_key_version: "encryptionkey@0"
I20260807 20:13:05.720772 32246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:05.724035 32246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:05.724257 32246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:05.724350 32246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:05.724509 32246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:05.724761 32246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:05.724789 32246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:05.724813 32246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:05.724829 32246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:05.730182 32246 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:43513
I20260807 20:13:05.730225 32443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:43513 every 8 connection(s)
I20260807 20:13:05.730518 32246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:13:05.733372 31762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.731193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:38761 (local address 127.23.200.254:37125)
0807 20:13:05.731269 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:05.731271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:05.731490 (+ 219us) server_negotiation.cc:407] Connection header received
0807 20:13:05.731957 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:05.731959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:05.731965 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:05.732018 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:05.733162 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:05.733163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:05.733164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:05.733169 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:05.733185 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:05.733189 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:05.733241 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:05.733305 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 20:13:05.733318 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 20:13:05.733557 32446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.731172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37125 (local address 127.23.200.195:38761)
0807 20:13:05.731353 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:05.731388 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:05.731868 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:05.732079 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:05.732083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:05.732094 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:05.732991 (+ 897us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:13:05.732992 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:05.733023 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:13:05.733056 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:05.733246 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:05.733250 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:05.733253 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:13:05.733309 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 20:13:05.733361 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260807 20:13:05.734108 32444 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37125
I20260807 20:13:05.734189 32444 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:05.734311 32444 heartbeater.cc:511] Master 127.23.200.254:37125 requested a full tablet report, sending...
I20260807 20:13:05.734624 31699 ts_manager.cc:194] Registered new tserver with Master: 96f55e1632eb4aa6948056ec0c9af772 (127.23.200.195:43513)
I20260807 20:13:05.738909 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 32246
I20260807 20:13:05.738991 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:13:05.739154 24355 external_mini_cluster.cc:1428] Setting key ff7e401bd3ede0d86a027925f08bb4b5
I20260807 20:13:05.739923 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:13:05.742094 31762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.741187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53878 (local address 127.23.200.254:37125)
0807 20:13:05.741248 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:05.741250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:05.741356 (+ 106us) server_negotiation.cc:407] Connection header received
0807 20:13:05.741397 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:05.741398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:05.741403 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:05.741446 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:05.741940 (+ 494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:05.741940 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:05.741940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:05.741943 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:05.741954 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:05.741957 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:05.741988 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:05.742021 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 20:13:05.742032 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:13:05.743597 31699 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:13:05.745057 31699 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:13:05.748544 32452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.747052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:37673 (local address 127.0.0.1:33898)
0807 20:13:05.747164 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:05.747203 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:05.747254 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:05.747552 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:05.747561 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:05.747569 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:05.748154 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:13:05.748155 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:05.748169 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:13:05.748190 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:05.748412 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:05.748419 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:05.748422 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:13:05.748467 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 20:13:05.748480 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":54,"threads_started":1}
I20260807 20:13:05.748771 32453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.747148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33898 (local address 127.23.200.193:37673)
0807 20:13:05.747411 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:05.747417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:05.747428 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:13:05.747455 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:05.747458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:05.747468 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:05.747548 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:05.748270 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:05.748277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:05.748277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:05.748281 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:05.748315 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:05.748337 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:05.748646 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:05.748671 (+ 25us) server_negotiation.cc:299] Negotiation successful
0807 20:13:05.748685 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":90,"threads_started":1}
I20260807 20:13:05.749051 32454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.747248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:37693 (local address 127.0.0.1:52702)
0807 20:13:05.747363 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:05.747371 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:05.747404 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:05.747959 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:05.747960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:05.747963 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:05.748323 (+ 360us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:13:05.748324 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:05.748329 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:13:05.748338 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:05.748916 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:05.748917 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:05.748918 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 20:13:05.748968 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 20:13:05.748980 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":80,"threads_started":1}
I20260807 20:13:05.749070 32460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.747418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:43513 (local address 127.0.0.1:45048)
0807 20:13:05.747529 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:05.747537 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:05.747574 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:05.748382 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:05.748382 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:05.748386 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:05.748727 (+ 341us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:13:05.748727 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:05.748733 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:13:05.748744 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:05.748967 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:05.748968 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:05.748968 (+ 0us) client_negotiation.cc:769] Sending connection context
0807 20:13:05.748996 (+ 28us) client_negotiation.cc:240] Negotiation successful
0807 20:13:05.749006 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":64,"threads_started":1}
I20260807 20:13:05.749264 32456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.747350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52702 (local address 127.23.200.194:37693)
0807 20:13:05.747805 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:05.747809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:05.747819 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:13:05.747843 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:05.747845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:05.747854 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:05.747955 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:05.748430 (+ 475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:05.748438 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:05.748439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:05.748444 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:05.748481 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:05.748510 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:05.748594 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:05.749163 (+ 569us) server_negotiation.cc:299] Negotiation successful
0807 20:13:05.749182 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":84,"threads_started":1}
I20260807 20:13:05.749308 32461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.747605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45048 (local address 127.23.200.195:43513)
0807 20:13:05.748011 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:05.748015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:05.748024 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:05.748047 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:05.748050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:05.748058 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:05.748364 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:05.748818 (+ 454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:05.748821 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:05.748822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:05.748825 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:05.748851 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:05.748865 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:05.749175 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:05.749207 (+ 32us) server_negotiation.cc:299] Negotiation successful
0807 20:13:05.749222 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260807 20:13:05.749471 31912 tablet_service.cc:1467] Processing CreateTablet for tablet 2cca6554c36047bcb3ea5ff40f557d59 (DEFAULT_TABLE table=test-table [id=7ca149e559dd4833adf4fb3588e6fad3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:05.749862 31912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cca6554c36047bcb3ea5ff40f557d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:05.749884 32132 tablet_service.cc:1467] Processing CreateTablet for tablet 2cca6554c36047bcb3ea5ff40f557d59 (DEFAULT_TABLE table=test-table [id=7ca149e559dd4833adf4fb3588e6fad3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:05.749909 32345 tablet_service.cc:1467] Processing CreateTablet for tablet 2cca6554c36047bcb3ea5ff40f557d59 (DEFAULT_TABLE table=test-table [id=7ca149e559dd4833adf4fb3588e6fad3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:05.750222 32132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cca6554c36047bcb3ea5ff40f557d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:05.750237 32345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cca6554c36047bcb3ea5ff40f557d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:05.752609 32464 tablet_bootstrap.cc:492] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772: Bootstrap starting.
I20260807 20:13:05.753188 32465 tablet_bootstrap.cc:492] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62: Bootstrap starting.
I20260807 20:13:05.753355 32464 tablet_bootstrap.cc:654] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:05.753754 32464 log.cc:824] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:05.753836 32465 tablet_bootstrap.cc:654] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:05.754112 32465 log.cc:824] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:05.754439 32466 tablet_bootstrap.cc:492] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d: Bootstrap starting.
I20260807 20:13:05.754535 32464 tablet_bootstrap.cc:492] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772: No bootstrap required, opened a new log
I20260807 20:13:05.754597 32464 ts_tablet_manager.cc:1397] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:13:05.754788 32465 tablet_bootstrap.cc:492] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62: No bootstrap required, opened a new log
I20260807 20:13:05.754848 32465 ts_tablet_manager.cc:1397] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:13:05.755053 32466 tablet_bootstrap.cc:654] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:05.755352 32466 log.cc:824] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:05.756085 32466 tablet_bootstrap.cc:492] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d: No bootstrap required, opened a new log
I20260807 20:13:05.756157 32466 ts_tablet_manager.cc:1397] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:13:05.756850 32464 raft_consensus.cc:348] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37673 } } peers { permanent_uuid: "e21c43913f5f4e87a0039c3fed88ef62" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 37693 } } peers { permanent_uuid: "96f55e1632eb4aa6948056ec0c9af772" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43513 } }
I20260807 20:13:05.756989 32464 raft_consensus.cc:374] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:05.757020 32464 raft_consensus.cc:729] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f55e1632eb4aa6948056ec0c9af772, State: Initialized, Role: FOLLOWER
I20260807 20:13:05.757177 32464 consensus_queue.cc:260] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37673 } } peers { permanent_uuid: "e21c43913f5f4e87a0039c3fed88ef62" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 37693 } } peers { permanent_uuid: "96f55e1632eb4aa6948056ec0c9af772" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43513 } }
I20260807 20:13:05.757282 32465 raft_consensus.cc:348] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37673 } } peers { permanent_uuid: "e21c43913f5f4e87a0039c3fed88ef62" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 37693 } } peers { permanent_uuid: "96f55e1632eb4aa6948056ec0c9af772" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43513 } }
I20260807 20:13:05.757423 32464 ts_tablet_manager.cc:1428] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:13:05.757444 32465 raft_consensus.cc:374] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:05.757490 32465 raft_consensus.cc:729] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e21c43913f5f4e87a0039c3fed88ef62, State: Initialized, Role: FOLLOWER
I20260807 20:13:05.757550 32444 heartbeater.cc:503] Master 127.23.200.254:37125 was elected leader, sending a full tablet report...
I20260807 20:13:05.757603 32465 consensus_queue.cc:260] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37673 } } peers { permanent_uuid: "e21c43913f5f4e87a0039c3fed88ef62" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 37693 } } peers { permanent_uuid: "96f55e1632eb4aa6948056ec0c9af772" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43513 } }
I20260807 20:13:05.757853 32465 ts_tablet_manager.cc:1428] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:13:05.758036 32240 heartbeater.cc:503] Master 127.23.200.254:37125 was elected leader, sending a full tablet report...
I20260807 20:13:05.758636 32466 raft_consensus.cc:348] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37673 } } peers { permanent_uuid: "e21c43913f5f4e87a0039c3fed88ef62" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 37693 } } peers { permanent_uuid: "96f55e1632eb4aa6948056ec0c9af772" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43513 } }
I20260807 20:13:05.758790 32466 raft_consensus.cc:374] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:05.758826 32466 raft_consensus.cc:729] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cd53247f4624ea7bb6a4c6aba9bef3d, State: Initialized, Role: FOLLOWER
I20260807 20:13:05.758934 32466 consensus_queue.cc:260] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37673 } } peers { permanent_uuid: "e21c43913f5f4e87a0039c3fed88ef62" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 37693 } } peers { permanent_uuid: "96f55e1632eb4aa6948056ec0c9af772" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43513 } }
I20260807 20:13:05.759148 32466 ts_tablet_manager.cc:1428] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:13:05.759191 32017 heartbeater.cc:503] Master 127.23.200.254:37125 was elected leader, sending a full tablet report...
I20260807 20:13:05.776247 32476 raft_consensus.cc:482] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:13:05.776368 32476 raft_consensus.cc:504] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37673 } } peers { permanent_uuid: "e21c43913f5f4e87a0039c3fed88ef62" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 37693 } } peers { permanent_uuid: "96f55e1632eb4aa6948056ec0c9af772" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43513 } }
I20260807 20:13:05.776678 32476 leader_election.cc:288] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e21c43913f5f4e87a0039c3fed88ef62 (127.23.200.194:37693), 96f55e1632eb4aa6948056ec0c9af772 (127.23.200.195:43513)
I20260807 20:13:05.777971 32019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.776861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:37693 (local address 127.23.200.193:49669)
0807 20:13:05.776922 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:05.776934 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:05.776975 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:05.777169 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:05.777170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:05.777173 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:05.777658 (+ 485us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:13:05.777659 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:05.777667 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:13:05.777681 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:05.777831 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:05.777832 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:05.777833 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 20:13:05.777890 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 20:13:05.777905 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 20:13:05.777994 32456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.776951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:49669 (local address 127.23.200.194:37693)
0807 20:13:05.777019 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:05.777021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:05.777030 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:05.777058 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:05.777059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:05.777064 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:05.777150 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:05.777746 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:05.777748 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:05.777748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:05.777753 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:05.777772 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:05.777776 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:05.777831 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:05.777904 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 20:13:05.777920 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 20:13:05.778326 32477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.776961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:43513 (local address 127.23.200.193:33597)
0807 20:13:05.777160 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:05.777171 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:05.777223 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:05.777343 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:05.777343 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:05.777345 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:05.777883 (+ 538us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:13:05.777883 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:05.777889 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:13:05.777899 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:05.778157 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:05.778158 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:05.778158 (+ 0us) client_negotiation.cc:769] Sending connection context
0807 20:13:05.778235 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 20:13:05.778248 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":148,"threads_started":1}
I20260807 20:13:05.778319 32461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.777042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:33597 (local address 127.23.200.195:43513)
0807 20:13:05.777131 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:05.777133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:05.777225 (+ 92us) server_negotiation.cc:407] Connection header received
0807 20:13:05.777270 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:05.777271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:05.777276 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:05.777330 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:05.777957 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:05.777959 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:05.777959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:05.777963 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:05.777980 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:05.777984 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:05.778039 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:05.778243 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 20:13:05.778258 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:13:05.778316 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cca6554c36047bcb3ea5ff40f557d59" candidate_uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e21c43913f5f4e87a0039c3fed88ef62" is_pre_election: true
I20260807 20:13:05.778486 32168 raft_consensus.cc:2454] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cd53247f4624ea7bb6a4c6aba9bef3d in term 0.
I20260807 20:13:05.778720 31826 leader_election.cc:302] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cd53247f4624ea7bb6a4c6aba9bef3d, e21c43913f5f4e87a0039c3fed88ef62; no voters:
I20260807 20:13:05.778842 32476 raft_consensus.cc:2781] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:13:05.778911 32476 raft_consensus.cc:482] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:13:05.778935 32476 raft_consensus.cc:3037] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:05.779052 32377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cca6554c36047bcb3ea5ff40f557d59" candidate_uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f55e1632eb4aa6948056ec0c9af772" is_pre_election: true
I20260807 20:13:05.779183 32377 raft_consensus.cc:2454] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cd53247f4624ea7bb6a4c6aba9bef3d in term 0.
I20260807 20:13:05.779717 32476 raft_consensus.cc:504] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37673 } } peers { permanent_uuid: "e21c43913f5f4e87a0039c3fed88ef62" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 37693 } } peers { permanent_uuid: "96f55e1632eb4aa6948056ec0c9af772" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43513 } }
I20260807 20:13:05.779829 32476 leader_election.cc:288] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [CANDIDATE]: Term 1 election: Requested vote from peers e21c43913f5f4e87a0039c3fed88ef62 (127.23.200.194:37693), 96f55e1632eb4aa6948056ec0c9af772 (127.23.200.195:43513)
I20260807 20:13:05.779999 32377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cca6554c36047bcb3ea5ff40f557d59" candidate_uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96f55e1632eb4aa6948056ec0c9af772"
I20260807 20:13:05.780061 32377 raft_consensus.cc:3037] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:05.780777 32377 raft_consensus.cc:2454] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cd53247f4624ea7bb6a4c6aba9bef3d in term 1.
I20260807 20:13:05.780941 31826 leader_election.cc:302] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96f55e1632eb4aa6948056ec0c9af772, 9cd53247f4624ea7bb6a4c6aba9bef3d; no voters:
I20260807 20:13:05.780912 32168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cca6554c36047bcb3ea5ff40f557d59" candidate_uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e21c43913f5f4e87a0039c3fed88ef62"
I20260807 20:13:05.781020 32476 raft_consensus.cc:2781] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:05.781016 32168 raft_consensus.cc:3037] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:05.781203 32476 raft_consensus.cc:686] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [term 1 LEADER]: Becoming Leader. State: Replica: 9cd53247f4624ea7bb6a4c6aba9bef3d, State: Running, Role: LEADER
I20260807 20:13:05.781301 32476 consensus_queue.cc:237] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37673 } } peers { permanent_uuid: "e21c43913f5f4e87a0039c3fed88ef62" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 37693 } } peers { permanent_uuid: "96f55e1632eb4aa6948056ec0c9af772" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43513 } }
I20260807 20:13:05.781854 32168 raft_consensus.cc:2454] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cd53247f4624ea7bb6a4c6aba9bef3d in term 1.
I20260807 20:13:05.782140 31698 catalog_manager.cc:5495] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cd53247f4624ea7bb6a4c6aba9bef3d (127.23.200.193). New cstate: current_term: 1 leader_uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cd53247f4624ea7bb6a4c6aba9bef3d" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e21c43913f5f4e87a0039c3fed88ef62" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 37693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96f55e1632eb4aa6948056ec0c9af772" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43513 } health_report { overall_health: UNKNOWN } } }
I20260807 20:13:05.788132 32453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.787364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33908 (local address 127.23.200.193:37673)
0807 20:13:05.787423 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:05.787424 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:05.787445 (+ 21us) server_negotiation.cc:407] Connection header received
0807 20:13:05.787473 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:05.787473 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:05.787484 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:05.787522 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:05.787958 (+ 436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:05.787959 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:05.787959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:05.787963 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:05.787977 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:05.787980 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:05.788022 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:05.788055 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 20:13:05.788065 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:13:05.789774 32168 raft_consensus.cc:1261] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62 [term 1 FOLLOWER]: Refusing update from remote peer 9cd53247f4624ea7bb6a4c6aba9bef3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:05.789776 32377 raft_consensus.cc:1261] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772 [term 1 FOLLOWER]: Refusing update from remote peer 9cd53247f4624ea7bb6a4c6aba9bef3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:05.790107 32480 consensus_queue.cc:1035] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e21c43913f5f4e87a0039c3fed88ef62" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 37693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:05.790231 32476 consensus_queue.cc:1035] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "96f55e1632eb4aa6948056ec0c9af772" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:05.793066 32487 mvcc.cc:205] Tried to move back new op lower bound from 7316003167392391168 to 7316003167360786432. Current Snapshot: MvccSnapshot[applied={T|T < 7316003167392391168 or (T in {7316003167392391168})}]
I20260807 20:13:05.795058 31699 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:05.795145 31699 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:05.795826 31699 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c24c789eaf0487887e4906d38598397: Sending DeleteTablet for 3 replicas of tablet 2cca6554c36047bcb3ea5ff40f557d59
I20260807 20:13:05.796134 31912 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cca6554c36047bcb3ea5ff40f557d59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:05 UTC) from {username='slave'} at 127.0.0.1:33898
I20260807 20:13:05.796224 32132 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cca6554c36047bcb3ea5ff40f557d59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:05 UTC) from {username='slave'} at 127.0.0.1:52702
I20260807 20:13:05.796379 32503 tablet_replica.cc:330] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d: stopping tablet replica
I20260807 20:13:05.796437 32345 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cca6554c36047bcb3ea5ff40f557d59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:05 UTC) from {username='slave'} at 127.0.0.1:45048
I20260807 20:13:05.796485 32503 raft_consensus.cc:2229] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:13:05.796521 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 31799
I20260807 20:13:05.796571 32503 raft_consensus.cc:2258] T 2cca6554c36047bcb3ea5ff40f557d59 P 9cd53247f4624ea7bb6a4c6aba9bef3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:05.806942 32504 tablet_replica.cc:330] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62: stopping tablet replica
I20260807 20:13:05.807068 32504 raft_consensus.cc:2229] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:05.807224 32504 raft_consensus.cc:2258] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:05.807576 32506 tablet_replica.cc:330] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772: stopping tablet replica
I20260807 20:13:05.807722 32506 raft_consensus.cc:2229] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:05.807843 32506 raft_consensus.cc:2258] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:05.810022 32504 ts_tablet_manager.cc:1905] T 2cca6554c36047bcb3ea5ff40f557d59 P e21c43913f5f4e87a0039c3fed88ef62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:05.810191 32506 ts_tablet_manager.cc:1905] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:05.812124 32506 ts_tablet_manager.cc:1918] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:13:05.812207 32506 log.cc:1196] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/2cca6554c36047bcb3ea5ff40f557d59
W20260807 20:13:05.812328 31672 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:37673 (error 108)
I20260807 20:13:05.812475 32506 ts_tablet_manager.cc:1939] T 2cca6554c36047bcb3ea5ff40f557d59 P 96f55e1632eb4aa6948056ec0c9af772: Deleting consensus metadata
I20260807 20:13:05.812506 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 32020
I20260807 20:13:05.820454 31672 catalog_manager.cc:4841] TS 96f55e1632eb4aa6948056ec0c9af772 (127.23.200.195:43513): tablet 2cca6554c36047bcb3ea5ff40f557d59 (table test-table [id=7ca149e559dd4833adf4fb3588e6fad3]) successfully deleted
I20260807 20:13:05.820590 32460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.820386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:37673 (local address 127.0.0.1:33918)
0807 20:13:05.820473 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:05.820544 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:37673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 20:13:05.821199 31672 catalog_manager.cc:4584] TS 9cd53247f4624ea7bb6a4c6aba9bef3d (127.23.200.193:37673): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2cca6554c36047bcb3ea5ff40f557d59: Network error: Client connection negotiation failed: client connection to 127.23.200.193:37673: connect: Connection refused (error 111)
I20260807 20:13:05.825387 32460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.825279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:37693 (local address 127.0.0.1:52704)
0807 20:13:05.825326 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:05.825358 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.194:37693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 20:13:05.825407 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 32246
I20260807 20:13:05.839113 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 31653
2026-08-07T20:13:05Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (681 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 20:13:05.864279 24355 mini_oidc.cc:152] Starting JWKS server
I20260807 20:13:05.865684 24355 webserver.cc:460] Webserver started at https://127.0.0.1:35295/ using document root <none> and password file <none>
I20260807 20:13:05.865777 24355 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 20:13:05.865969 24355 webserver.cc:460] Webserver started at http://127.0.0.1:36591/ using document root <none> and password file <none>
2026-08-07T20:13:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:05Z Disabled control of system clock
I20260807 20:13:05.874836 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:42491
--webserver_interface=127.23.200.254
--webserver_port=0
--builtin_ntp_servers=127.23.200.212:41373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:42491
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35295/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/testchainca.cert with env {}
W20260807 20:13:05.956184 32515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:05.956394 32515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:05.956415 32515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:05.957552 32515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:05.957588 32515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:05.957600 32515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:05.957612 32515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:05.957630 32515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 20:13:05.957652 32515 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 20:13:05.957662 32515 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35295/jwks.json
I20260807 20:13:05.959233 32515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:41373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:42491
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:42491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35295/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:05.959440 32515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:05.959613 32515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:05.964409 32524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:05.964545 32515 server_base.cc:1034] running on GCE node
W20260807 20:13:05.964408 32528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:05.964386 32523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:05.964897 32515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:05.965214 32515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:05.966364 32515 hybrid_clock.cc:648] HybridClock initialized: now 1786133585966344 us; error 28 us; skew 500 ppm
I20260807 20:13:05.967628 32515 webserver.cc:460] Webserver started at http://127.23.200.254:43683/ using document root <none> and password file <none>
I20260807 20:13:05.967849 32515 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:05.967906 32515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:05.968034 32515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:05.969151 32515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "bdb8883e924a46268a33fdb0b155b832"
format_stamp: "Formatted at 2026-08-07 20:13:05 on dist-test-slave-2j7r"
server_key: "9609a62400b5996f7a5859d0d7ad1f42"
server_key_iv: "83fa8a7527a3606635222f415ba67885"
server_key_version: "encryptionkey@0"
I20260807 20:13:05.969509 32515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "bdb8883e924a46268a33fdb0b155b832"
format_stamp: "Formatted at 2026-08-07 20:13:05 on dist-test-slave-2j7r"
server_key: "9609a62400b5996f7a5859d0d7ad1f42"
server_key_iv: "83fa8a7527a3606635222f415ba67885"
server_key_version: "encryptionkey@0"
I20260807 20:13:05.970762 32515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:13:05.971457 32539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:05.971607 32515 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:13:05.971691 32515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "bdb8883e924a46268a33fdb0b155b832"
format_stamp: "Formatted at 2026-08-07 20:13:05 on dist-test-slave-2j7r"
server_key: "9609a62400b5996f7a5859d0d7ad1f42"
server_key_iv: "83fa8a7527a3606635222f415ba67885"
server_key_version: "encryptionkey@0"
I20260807 20:13:05.971752 32515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:05.992923 32515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:05.993283 32515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:05.993436 32515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:05.997219 32515 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:42491
I20260807 20:13:05.997272 32620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:42491 every 8 connection(s)
I20260807 20:13:05.997609 32515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:13:05.998553 32621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:05.999099 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 32515
I20260807 20:13:05.999192 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:13:05.999413 24355 external_mini_cluster.cc:1428] Setting key bc238c0e2a9fb345507273fafd873568
I20260807 20:13:06.001663 32621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832: Bootstrap starting.
I20260807 20:13:06.002403 32621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:06.002705 32621 log.cc:824] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:06.003399 32621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832: No bootstrap required, opened a new log
I20260807 20:13:06.005530 32626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:05.999950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33084 (local address 127.23.200.254:42491)
0807 20:13:06.000196 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:06.000206 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:06.000241 (+ 35us) server_negotiation.cc:407] Connection header received
0807 20:13:06.000783 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:06.000793 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:06.000880 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:06.001038 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:06.001272 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.002552 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.003330 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.003921 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.004047 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:06.004741 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:06.004745 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:06.004746 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:06.004968 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:06.005017 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:06.005034 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:06.005132 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:06.005223 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 20:13:06.005319 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":103,"threads_started":1}
I20260807 20:13:06.005599 32621 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb8883e924a46268a33fdb0b155b832" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42491 } }
I20260807 20:13:06.005751 32621 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:06.005785 32621 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdb8883e924a46268a33fdb0b155b832, State: Initialized, Role: FOLLOWER
I20260807 20:13:06.005911 32621 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb8883e924a46268a33fdb0b155b832" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42491 } }
I20260807 20:13:06.005999 32621 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:13:06.006032 32621 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:13:06.006080 32621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:06.006558 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:13:06.006765 32621 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb8883e924a46268a33fdb0b155b832" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42491 } }
I20260807 20:13:06.006870 32621 leader_election.cc:302] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdb8883e924a46268a33fdb0b155b832; no voters:
I20260807 20:13:06.007026 32621 leader_election.cc:288] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:13:06.007071 32628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:06.007269 32628 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [term 1 LEADER]: Becoming Leader. State: Replica: bdb8883e924a46268a33fdb0b155b832, State: Running, Role: LEADER
I20260807 20:13:06.007310 32621 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:06.007366 32628 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb8883e924a46268a33fdb0b155b832" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42491 } }
I20260807 20:13:06.007679 32628 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdb8883e924a46268a33fdb0b155b832. Latest consensus state: current_term: 1 leader_uuid: "bdb8883e924a46268a33fdb0b155b832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb8883e924a46268a33fdb0b155b832" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42491 } } }
I20260807 20:13:06.007766 32628 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:06.007692 32632 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdb8883e924a46268a33fdb0b155b832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb8883e924a46268a33fdb0b155b832" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42491 } } }
I20260807 20:13:06.007829 32632 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:06.007988 32639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:13:06.008564 32639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:06.010201 32639 catalog_manager.cc:1347] Generated new cluster ID: 1fcce7e9b3e94c30a4a83d8a5f2cb9bc
I20260807 20:13:06.010254 32639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:13:06.028523 32639 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:13:06.029048 32639 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:06.034761 32639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdb8883e924a46268a33fdb0b155b832: Generated new TSK 0
I20260807 20:13:06.034965 32639 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 20:13:06.062152 32673 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 20:13:06.062318 32673 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 20:13:06.062456 32626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:06.046241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33098 (local address 127.23.200.254:42491)
0807 20:13:06.046289 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:06.046292 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:06.046418 (+ 126us) server_negotiation.cc:407] Connection header received
0807 20:13:06.046508 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:06.046509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:06.046519 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:06.046575 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0807 20:13:06.046711 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.047192 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.048300 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.048540 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.048608 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:06.048848 (+ 240us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 20:13:06.062193 (+ 13345us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35295/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 20:13:06.062384 (+ 191us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33098: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35295/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 20:13:06.062515 32626 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33098: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35295/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 20:13:06.063171 32671 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:06.046158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42491 (local address 127.0.0.1:33098)
0807 20:13:06.046329 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:06.046340 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:06.046421 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:06.046580 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:06.046581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:06.046583 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 20:13:06.046656 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:06.046659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.047275 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.047276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:06.048205 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:06.048208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.048626 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.048627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:06.048692 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:06.048717 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 20:13:06.062584 (+ 13867us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35295/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 20:13:06.062728 (+ 144us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.200.254:42491: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35295/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.23.200.254:42491: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35295/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 20:13:06.063688 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 32515
2026-08-07T20:13:06Z chronyd exiting
I20260807 20:13:06.108594 24355 test_util.cc:175] -----------------------------------------------
I20260807 20:13:06.108673 24355 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133562076833-24355-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (245 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 20:13:06.109486 24355 mini_oidc.cc:152] Starting JWKS server
I20260807 20:13:06.110146 24355 webserver.cc:460] Webserver started at https://127.0.0.1:42253/ using document root <none> and password file <none>
I20260807 20:13:06.110189 24355 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 20:13:06.110337 24355 webserver.cc:460] Webserver started at http://127.0.0.1:44539/ using document root <none> and password file <none>
2026-08-07T20:13:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:06Z Disabled control of system clock
I20260807 20:13:06.119107 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:33083
--webserver_interface=127.23.200.254
--webserver_port=0
--builtin_ntp_servers=127.23.200.212:35683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:33083
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42253/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/testchainca.cert with env {}
W20260807 20:13:06.200520 32683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:06.200809 32683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:06.200835 32683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:06.202044 32683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:06.202085 32683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:06.202095 32683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:06.202107 32683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:06.202118 32683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 20:13:06.202132 32683 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 20:13:06.202143 32683 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42253/jwks.json
I20260807 20:13:06.203677 32683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:35683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:33083
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:33083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42253/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:06.203896 32683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:06.204077 32683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:06.208902 32691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:06.208913 32693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:06.209046 32683 server_base.cc:1034] running on GCE node
W20260807 20:13:06.208918 32689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:06.209407 32683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:06.209728 32683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:06.210897 32683 hybrid_clock.cc:648] HybridClock initialized: now 1786133586210874 us; error 39 us; skew 500 ppm
I20260807 20:13:06.212123 32683 webserver.cc:460] Webserver started at http://127.23.200.254:45913/ using document root <none> and password file <none>
I20260807 20:13:06.212349 32683 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:06.212415 32683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:06.212551 32683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:06.213675 32683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "a1b96c910ae24874993f94eb7703da3f"
format_stamp: "Formatted at 2026-08-07 20:13:06 on dist-test-slave-2j7r"
server_key: "94df92bc6c41a4860f72111afd529e61"
server_key_iv: "d2d2d7b81fac9fdf6b25614aabe7172f"
server_key_version: "encryptionkey@0"
I20260807 20:13:06.214015 32683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "a1b96c910ae24874993f94eb7703da3f"
format_stamp: "Formatted at 2026-08-07 20:13:06 on dist-test-slave-2j7r"
server_key: "94df92bc6c41a4860f72111afd529e61"
server_key_iv: "d2d2d7b81fac9fdf6b25614aabe7172f"
server_key_version: "encryptionkey@0"
I20260807 20:13:06.215250 32683 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 20:13:06.215965 32704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:06.216097 32683 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:06.216181 32683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "a1b96c910ae24874993f94eb7703da3f"
format_stamp: "Formatted at 2026-08-07 20:13:06 on dist-test-slave-2j7r"
server_key: "94df92bc6c41a4860f72111afd529e61"
server_key_iv: "d2d2d7b81fac9fdf6b25614aabe7172f"
server_key_version: "encryptionkey@0"
I20260807 20:13:06.216250 32683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:06.225175 32683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:06.225476 32683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:06.225596 32683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:06.229384 32683 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:33083
I20260807 20:13:06.229446 321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:33083 every 8 connection(s)
I20260807 20:13:06.229773 32683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:13:06.230475 322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:06.233335 322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f: Bootstrap starting.
I20260807 20:13:06.233574 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 32683
I20260807 20:13:06.233667 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:13:06.233834 24355 external_mini_cluster.cc:1428] Setting key bef5b896466b8eac25583b30d778b44b
I20260807 20:13:06.234002 322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:06.234297 322 log.cc:824] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:06.235088 322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f: No bootstrap required, opened a new log
I20260807 20:13:06.237459 322 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b96c910ae24874993f94eb7703da3f" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33083 } }
I20260807 20:13:06.237600 322 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:06.237622 322 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b96c910ae24874993f94eb7703da3f, State: Initialized, Role: FOLLOWER
I20260807 20:13:06.237736 322 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b96c910ae24874993f94eb7703da3f" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33083 } }
I20260807 20:13:06.237807 322 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:13:06.237830 322 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:13:06.237875 322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:06.238600 322 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b96c910ae24874993f94eb7703da3f" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33083 } }
I20260807 20:13:06.238710 322 leader_election.cc:302] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1b96c910ae24874993f94eb7703da3f; no voters:
I20260807 20:13:06.238868 322 leader_election.cc:288] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:13:06.238889 331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:06.238998 331 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [term 1 LEADER]: Becoming Leader. State: Replica: a1b96c910ae24874993f94eb7703da3f, State: Running, Role: LEADER
I20260807 20:13:06.239073 331 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b96c910ae24874993f94eb7703da3f" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33083 } }
I20260807 20:13:06.239140 322 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:06.239394 331 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1b96c910ae24874993f94eb7703da3f. Latest consensus state: current_term: 1 leader_uuid: "a1b96c910ae24874993f94eb7703da3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b96c910ae24874993f94eb7703da3f" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33083 } } }
I20260807 20:13:06.239465 331 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:06.240190 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:06.234325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36570 (local address 127.23.200.254:33083)
0807 20:13:06.234543 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:06.234552 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:06.234581 (+ 29us) server_negotiation.cc:407] Connection header received
0807 20:13:06.235155 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:06.235166 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:06.235252 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:06.235408 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:06.235571 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.236942 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.237960 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.238551 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.238687 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:06.239356 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:06.239361 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:06.239361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:06.239576 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:06.239620 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:06.239636 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:06.239692 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:06.239897 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 20:13:06.239989 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260807 20:13:06.240345 336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:13:06.240538 332 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1b96c910ae24874993f94eb7703da3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b96c910ae24874993f94eb7703da3f" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33083 } } }
I20260807 20:13:06.240927 332 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:06.240829 336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:06.241398 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:13:06.242552 336 catalog_manager.cc:1347] Generated new cluster ID: 7ceff15dbe2a4682b579158d34e1f476
I20260807 20:13:06.242599 336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:13:06.268714 336 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:13:06.269268 336 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:06.271574 336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1b96c910ae24874993f94eb7703da3f: Generated new TSK 0
I20260807 20:13:06.271737 336 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 20:13:06.299357 377 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 20:13:06.299461 377 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 20:13:06.299782 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:06.279572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36574 (local address 127.23.200.254:33083)
0807 20:13:06.279624 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:06.279626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:06.279743 (+ 117us) server_negotiation.cc:407] Connection header received
0807 20:13:06.279806 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:06.279807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:06.279819 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:06.279885 (+ 66us) server_negotiation.cc:226] Negotiated authn=JWT
0807 20:13:06.280019 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.280510 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.281345 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.281553 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.281606 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:06.281776 (+ 170us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 20:13:06.299556 (+ 17780us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42253/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 20:13:06.299712 (+ 156us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36574: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42253/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":26}
W20260807 20:13:06.299844 328 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36574: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42253/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 20:13:06.299875 370 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:06.279498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:33083 (local address 127.0.0.1:36574)
0807 20:13:06.279667 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:06.279676 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:06.279736 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:06.279887 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:06.279888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:06.279890 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 20:13:06.279959 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:06.279961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.280601 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.280602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:06.281268 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:06.281271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.281614 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.281615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:06.281681 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:06.281687 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 20:13:06.299669 (+ 17982us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42253/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 20:13:06.299783 (+ 114us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.200.254:33083: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42253/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
I20260807 20:13:06.300118 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 32683
2026-08-07T20:13:06Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (239 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 20:13:06.348613 24355 mini_oidc.cc:152] Starting JWKS server
I20260807 20:13:06.349350 24355 webserver.cc:460] Webserver started at https://127.0.0.1:35205/ using document root <none> and password file <none>
I20260807 20:13:06.349407 24355 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 20:13:06.349545 24355 webserver.cc:460] Webserver started at http://127.0.0.1:34899/ using document root <none> and password file <none>
2026-08-07T20:13:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:06Z Disabled control of system clock
I20260807 20:13:06.358457 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:43153
--webserver_interface=127.23.200.254
--webserver_port=0
--builtin_ntp_servers=127.23.200.212:44007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:43153
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35205/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 20:13:06.442029 391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:06.442323 391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:06.442358 391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:06.443807 391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:06.443856 391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:06.443867 391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:06.443879 391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:06.443898 391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 20:13:06.443910 391 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 20:13:06.443928 391 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35205/jwks.json
I20260807 20:13:06.445380 391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:43153
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:43153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35205/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:06.445608 391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:06.445866 391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:06.450794 397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:06.450866 405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:06.451035 402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:06.451306 391 server_base.cc:1034] running on GCE node
I20260807 20:13:06.451577 391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:06.451870 391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:06.453033 391 hybrid_clock.cc:648] HybridClock initialized: now 1786133586453016 us; error 27 us; skew 500 ppm
I20260807 20:13:06.454270 391 webserver.cc:460] Webserver started at http://127.23.200.254:34643/ using document root <none> and password file <none>
I20260807 20:13:06.454511 391 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:06.454571 391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:06.454707 391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:06.455849 391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "57a7d98fd5974dd292ced8a585a14e76"
format_stamp: "Formatted at 2026-08-07 20:13:06 on dist-test-slave-2j7r"
server_key: "ea3359236c3674e93e3ace5b3f6365df"
server_key_iv: "25d628c40e8ece61a5f84d504b2f8f42"
server_key_version: "encryptionkey@0"
I20260807 20:13:06.456185 391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "57a7d98fd5974dd292ced8a585a14e76"
format_stamp: "Formatted at 2026-08-07 20:13:06 on dist-test-slave-2j7r"
server_key: "ea3359236c3674e93e3ace5b3f6365df"
server_key_iv: "25d628c40e8ece61a5f84d504b2f8f42"
server_key_version: "encryptionkey@0"
I20260807 20:13:06.457509 391 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:13:06.458237 414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:06.458396 391 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:06.458508 391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "57a7d98fd5974dd292ced8a585a14e76"
format_stamp: "Formatted at 2026-08-07 20:13:06 on dist-test-slave-2j7r"
server_key: "ea3359236c3674e93e3ace5b3f6365df"
server_key_iv: "25d628c40e8ece61a5f84d504b2f8f42"
server_key_version: "encryptionkey@0"
I20260807 20:13:06.458572 391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:06.481805 391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:06.482159 391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:06.482316 391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:06.486403 391 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:43153
I20260807 20:13:06.486465 501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:43153 every 8 connection(s)
I20260807 20:13:06.486806 391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:13:06.487540 502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:06.490315 502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76: Bootstrap starting.
I20260807 20:13:06.490904 502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:06.491138 502 log.cc:824] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:06.491885 502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76: No bootstrap required, opened a new log
I20260807 20:13:06.493206 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 391
I20260807 20:13:06.493294 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:13:06.493465 24355 external_mini_cluster.cc:1428] Setting key c0197309461c5ec31410e47115494ff5
I20260807 20:13:06.493577 502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57a7d98fd5974dd292ced8a585a14e76" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 43153 } }
I20260807 20:13:06.493763 502 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:06.493819 502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57a7d98fd5974dd292ced8a585a14e76, State: Initialized, Role: FOLLOWER
I20260807 20:13:06.493970 502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57a7d98fd5974dd292ced8a585a14e76" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 43153 } }
I20260807 20:13:06.494061 502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:13:06.494096 502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:13:06.494146 502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:06.494714 502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57a7d98fd5974dd292ced8a585a14e76" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 43153 } }
I20260807 20:13:06.494829 502 leader_election.cc:302] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57a7d98fd5974dd292ced8a585a14e76; no voters:
I20260807 20:13:06.494997 502 leader_election.cc:288] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:13:06.495203 509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:06.495257 502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:06.495350 509 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [term 1 LEADER]: Becoming Leader. State: Replica: 57a7d98fd5974dd292ced8a585a14e76, State: Running, Role: LEADER
I20260807 20:13:06.495448 509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57a7d98fd5974dd292ced8a585a14e76" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 43153 } }
I20260807 20:13:06.495813 509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57a7d98fd5974dd292ced8a585a14e76. Latest consensus state: current_term: 1 leader_uuid: "57a7d98fd5974dd292ced8a585a14e76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57a7d98fd5974dd292ced8a585a14e76" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 43153 } } }
I20260807 20:13:06.495821 512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57a7d98fd5974dd292ced8a585a14e76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57a7d98fd5974dd292ced8a585a14e76" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 43153 } } }
I20260807 20:13:06.495889 509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:06.495939 512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:06.496387 530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:13:06.496978 530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:06.499480 508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:06.494002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60970 (local address 127.23.200.254:43153)
0807 20:13:06.494254 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:06.494263 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:06.494288 (+ 25us) server_negotiation.cc:407] Connection header received
0807 20:13:06.494390 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:06.494395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:06.494502 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:06.494614 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:06.494863 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.496214 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.497264 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.497861 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.498009 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:06.498732 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:06.498736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:06.498736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:06.498958 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:06.499002 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:06.499017 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:06.499088 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:06.499173 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 20:13:06.499255 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":99,"threads_started":1}
I20260807 20:13:06.500037 530 catalog_manager.cc:1347] Generated new cluster ID: eb55fafde430491b855ed2d49bb594e9
I20260807 20:13:06.500092 530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:13:06.500377 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:13:06.536492 530 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:13:06.537048 530 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:06.546428 530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57a7d98fd5974dd292ced8a585a14e76: Generated new TSK 0
I20260807 20:13:06.546636 530 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:13:06.556844 508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:06.553379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60972 (local address 127.23.200.254:43153)
0807 20:13:06.553425 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:06.553427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:06.553520 (+ 93us) server_negotiation.cc:407] Connection header received
0807 20:13:06.553596 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:06.553597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:06.553607 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:06.553663 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0807 20:13:06.553813 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.554283 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.555418 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.555601 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.555651 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:06.555841 (+ 190us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 20:13:06.556633 (+ 792us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35205/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 20:13:06.556763 (+ 130us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60972: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35205/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 20:13:06.556932 508 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60972: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35205/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 20:13:06.557226 553 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:06.553287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:43153 (local address 127.0.0.1:60972)
0807 20:13:06.553446 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:06.553457 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:06.553531 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:06.553684 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:06.553685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:06.553687 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 20:13:06.553752 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:06.553754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.554373 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.554374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:06.555335 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:06.555339 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.555681 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.555683 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:06.555744 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:06.555750 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 20:13:06.556732 (+ 982us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35205/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 20:13:06.556841 (+ 109us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.200.254:43153: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35205/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260807 20:13:06.557699 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 391
E20260807 20:13:06.578420 554 webserver.cc:533] Webserver: sslize failed:
E20260807 20:13:06.578498 554 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T20:13:06Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (258 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T20:13:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:06Z Disabled control of system clock
I20260807 20:13:06.615113 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:35587
--webserver_interface=127.23.200.254
--webserver_port=0
--builtin_ntp_servers=127.23.200.212:44415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:35587
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/testchainca.cert with env {}
W20260807 20:13:06.698303 559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:06.698518 559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:06.698540 559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:06.699740 559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:06.699788 559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:06.699802 559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:06.699815 559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:06.699832 559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 20:13:06.699860 559 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 20:13:06.699885 559 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 20:13:06.701491 559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:35587
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:35587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:06.701756 559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:06.701973 559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:06.706871 572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:06.706866 568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:06.706961 559 server_base.cc:1034] running on GCE node
W20260807 20:13:06.706864 570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:06.707381 559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:06.707682 559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:06.708842 559 hybrid_clock.cc:648] HybridClock initialized: now 1786133586708818 us; error 31 us; skew 500 ppm
I20260807 20:13:06.710148 559 webserver.cc:460] Webserver started at http://127.23.200.254:42969/ using document root <none> and password file <none>
I20260807 20:13:06.710382 559 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:06.710456 559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:06.710583 559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:06.711618 559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "eba6fdde8dea42909e929025dadeb170"
format_stamp: "Formatted at 2026-08-07 20:13:06 on dist-test-slave-2j7r"
server_key: "9dc0eab83828318cd299727fed39b8b3"
server_key_iv: "0ceef12fe63edc7b8f553d5497b3bb6a"
server_key_version: "encryptionkey@0"
I20260807 20:13:06.711956 559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "eba6fdde8dea42909e929025dadeb170"
format_stamp: "Formatted at 2026-08-07 20:13:06 on dist-test-slave-2j7r"
server_key: "9dc0eab83828318cd299727fed39b8b3"
server_key_iv: "0ceef12fe63edc7b8f553d5497b3bb6a"
server_key_version: "encryptionkey@0"
I20260807 20:13:06.713174 559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:13:06.713851 579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:06.714004 559 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:06.714092 559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "eba6fdde8dea42909e929025dadeb170"
format_stamp: "Formatted at 2026-08-07 20:13:06 on dist-test-slave-2j7r"
server_key: "9dc0eab83828318cd299727fed39b8b3"
server_key_iv: "0ceef12fe63edc7b8f553d5497b3bb6a"
server_key_version: "encryptionkey@0"
I20260807 20:13:06.714157 559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:06.734906 559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:06.735226 559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:06.735360 559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:06.739205 559 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:35587
I20260807 20:13:06.739243 671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:35587 every 8 connection(s)
I20260807 20:13:06.739612 559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:13:06.740629 672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:06.742976 672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170: Bootstrap starting.
I20260807 20:13:06.743542 672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:06.743777 672 log.cc:824] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:06.744477 672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170: No bootstrap required, opened a new log
I20260807 20:13:06.746119 672 raft_consensus.cc:348] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba6fdde8dea42909e929025dadeb170" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 35587 } }
I20260807 20:13:06.746263 672 raft_consensus.cc:374] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:06.746286 672 raft_consensus.cc:729] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba6fdde8dea42909e929025dadeb170, State: Initialized, Role: FOLLOWER
I20260807 20:13:06.746394 672 consensus_queue.cc:260] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba6fdde8dea42909e929025dadeb170" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 35587 } }
I20260807 20:13:06.746444 672 raft_consensus.cc:388] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:13:06.746464 672 raft_consensus.cc:482] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:13:06.746501 672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:06.746977 672 raft_consensus.cc:504] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba6fdde8dea42909e929025dadeb170" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 35587 } }
I20260807 20:13:06.747061 672 leader_election.cc:302] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eba6fdde8dea42909e929025dadeb170; no voters:
I20260807 20:13:06.747179 672 leader_election.cc:288] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:13:06.747231 679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:06.747403 672 sys_catalog.cc:564] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:06.747416 679 raft_consensus.cc:686] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [term 1 LEADER]: Becoming Leader. State: Replica: eba6fdde8dea42909e929025dadeb170, State: Running, Role: LEADER
I20260807 20:13:06.747531 679 consensus_queue.cc:237] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba6fdde8dea42909e929025dadeb170" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 35587 } }
I20260807 20:13:06.747874 680 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eba6fdde8dea42909e929025dadeb170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba6fdde8dea42909e929025dadeb170" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 35587 } } }
I20260807 20:13:06.747972 680 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:06.748018 683 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eba6fdde8dea42909e929025dadeb170. Latest consensus state: current_term: 1 leader_uuid: "eba6fdde8dea42909e929025dadeb170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba6fdde8dea42909e929025dadeb170" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 35587 } } }
I20260807 20:13:06.748101 683 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:06.748337 691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:13:06.748818 691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:06.749718 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 559
I20260807 20:13:06.749832 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:13:06.749992 24355 external_mini_cluster.cc:1428] Setting key b7eac09212021ba6f8b35855c7139299
I20260807 20:13:06.750319 691 catalog_manager.cc:1347] Generated new cluster ID: 8ef1472f0d0b4ba58a668c2795fe6404
I20260807 20:13:06.750365 691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:13:06.758282 708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:06.750477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43800 (local address 127.23.200.254:35587)
0807 20:13:06.753252 (+ 2775us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:06.753261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:06.753295 (+ 34us) server_negotiation.cc:407] Connection header received
0807 20:13:06.753423 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:06.753431 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:06.753510 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:06.753656 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:06.753854 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.755253 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.756252 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.756891 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.757033 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:06.757598 (+ 565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:06.757602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:06.757603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:06.757807 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:06.757851 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:06.757867 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:06.757937 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:06.757992 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 20:13:06.758068 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2684,"thread_start_us":71,"threads_started":1}
I20260807 20:13:06.759199 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:13:06.766675 691 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:13:06.767154 691 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:06.772774 691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eba6fdde8dea42909e929025dadeb170: Generated new TSK 0
I20260807 20:13:06.773029 691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:13:07.253750 708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:06.777721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43810 (local address 127.23.200.254:35587)
0807 20:13:06.777766 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:06.777768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:06.777880 (+ 112us) server_negotiation.cc:407] Connection header received
0807 20:13:06.777967 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:06.777968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:06.777979 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:06.778038 (+ 59us) server_negotiation.cc:226] Negotiated authn=JWT
0807 20:13:06.778186 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.778693 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.779820 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.780035 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.780085 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:06.780262 (+ 177us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 20:13:07.253361 (+473099us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 20:13:07.253686 (+ 325us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43810: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 20:13:07.253820 708 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43810: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 20:13:07.253944 721 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:06.777639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:35587 (local address 127.0.0.1:43810)
0807 20:13:06.777815 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:06.777826 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:06.777898 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:06.778057 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:06.778058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:06.778061 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0807 20:13:06.778127 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:06.778129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.778768 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.778769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:06.779732 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:06.779735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:06.780107 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:06.780108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:06.780176 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:06.780182 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 20:13:07.253645 (+473463us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 20:13:07.253826 (+ 181us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.200.254:35587: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":69,"threads_started":1}
I20260807 20:13:07.254295 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 559
2026-08-07T20:13:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (673 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (93 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (92 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:13:07 dist-test-slave-2j7r krb5kdc[742](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:13:07 dist-test-slave-2j7r krb5kdc[742](info): set up 2 sockets
Aug 07 20:13:07 dist-test-slave-2j7r krb5kdc[742](info): commencing operation
krb5kdc: starting...
W20260807 20:13:08.640483 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:13:08 dist-test-slave-2j7r krb5kdc[742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:13:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:13:08.768551 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:46377
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:46377
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:08.852826 764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:08.853070 764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:08.853124 764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:08.854321 764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:08.854364 764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:08.854382 764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:08.854398 764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:08.854413 764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:13:08.855890 764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:46377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:46377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:08.856146 764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:08.856371 764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:08.861280 773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:08.861425 764 server_base.cc:1034] running on GCE node
W20260807 20:13:08.861289 775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:08.861528 772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:08.861891 764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:08.862202 764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:08.863361 764 hybrid_clock.cc:648] HybridClock initialized: now 1786133588863336 us; error 41 us; skew 500 ppm
Aug 07 20:13:08 dist-test-slave-2j7r krb5kdc[742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133588, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:08.866726 764 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:13:08.867280 764 webserver.cc:460] Webserver started at http://127.23.200.254:38155/ using document root <none> and password file <none>
I20260807 20:13:08.867558 764 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:08.867625 764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:08.867817 764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:08.869210 764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "28c7dd643bb94654ad33f54912e02c59"
format_stamp: "Formatted at 2026-08-07 20:13:08 on dist-test-slave-2j7r"
server_key: "b5195d0ebd2acee4dfdff41a1a71f9f3"
server_key_iv: "8c8284ec959ebfb178e813c30d75863a"
server_key_version: "encryptionkey@0"
I20260807 20:13:08.869639 764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "28c7dd643bb94654ad33f54912e02c59"
format_stamp: "Formatted at 2026-08-07 20:13:08 on dist-test-slave-2j7r"
server_key: "b5195d0ebd2acee4dfdff41a1a71f9f3"
server_key_iv: "8c8284ec959ebfb178e813c30d75863a"
server_key_version: "encryptionkey@0"
I20260807 20:13:08.871161 764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:08.871923 787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:08.872059 764 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:08.872152 764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "28c7dd643bb94654ad33f54912e02c59"
format_stamp: "Formatted at 2026-08-07 20:13:08 on dist-test-slave-2j7r"
server_key: "b5195d0ebd2acee4dfdff41a1a71f9f3"
server_key_iv: "8c8284ec959ebfb178e813c30d75863a"
server_key_version: "encryptionkey@0"
I20260807 20:13:08.872223 764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:08.889695 764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:08.892014 764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:08.892143 764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:08.896132 764 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:46377
I20260807 20:13:08.896176 873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:46377 every 8 connection(s)
I20260807 20:13:08.896524 764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:13:08.897452 874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:08.899837 874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59: Bootstrap starting.
I20260807 20:13:08.900447 874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:08.900728 874 log.cc:824] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:08.901453 874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59: No bootstrap required, opened a new log
I20260807 20:13:08.903064 874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c7dd643bb94654ad33f54912e02c59" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 46377 } }
I20260807 20:13:08.903213 874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:08.903235 874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28c7dd643bb94654ad33f54912e02c59, State: Initialized, Role: FOLLOWER
I20260807 20:13:08.903370 874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c7dd643bb94654ad33f54912e02c59" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 46377 } }
I20260807 20:13:08.903437 874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:13:08.903458 874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:13:08.903491 874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:08.904026 874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c7dd643bb94654ad33f54912e02c59" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 46377 } }
I20260807 20:13:08.904122 874 leader_election.cc:302] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28c7dd643bb94654ad33f54912e02c59; no voters:
I20260807 20:13:08.904299 874 leader_election.cc:288] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:13:08.904366 877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:08.904577 874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:08.904572 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 764
I20260807 20:13:08.904664 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:13:08.904752 877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [term 1 LEADER]: Becoming Leader. State: Replica: 28c7dd643bb94654ad33f54912e02c59, State: Running, Role: LEADER
I20260807 20:13:08.904874 24355 external_mini_cluster.cc:1428] Setting key 9f3377249700e4cef5f5de30305bd3d9
I20260807 20:13:08.904852 877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c7dd643bb94654ad33f54912e02c59" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 46377 } }
I20260807 20:13:08.905213 879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28c7dd643bb94654ad33f54912e02c59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c7dd643bb94654ad33f54912e02c59" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 46377 } } }
I20260807 20:13:08.905288 879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:08.905426 885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28c7dd643bb94654ad33f54912e02c59. Latest consensus state: current_term: 1 leader_uuid: "28c7dd643bb94654ad33f54912e02c59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c7dd643bb94654ad33f54912e02c59" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 46377 } } }
I20260807 20:13:08.905546 885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:08.905795 899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:13:08.906594 899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:08.908033 899 catalog_manager.cc:1347] Generated new cluster ID: 0427f683c8dc43509e08f9842b677025
I20260807 20:13:08.908082 899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:13:08 dist-test-slave-2j7r krb5kdc[742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:08.916467 904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:08.905590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33476 (local address 127.23.200.254:46377)
0807 20:13:08.905781 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:08.905792 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:08.905825 (+ 33us) server_negotiation.cc:407] Connection header received
0807 20:13:08.905977 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:08.905985 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:08.906047 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:08.906185 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:08.907055 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:08.908429 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:08.909477 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:08.910124 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:08.910245 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:08.912844 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:08.912849 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:08.912849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:08.912863 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:08.914830 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:08.915221 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:08.915225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:08.915225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:08.915343 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:08.915519 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:08.915520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:08.915520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:08.915841 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:08.915990 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:08.916072 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 20:13:08.916217 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
I20260807 20:13:08.920540 899 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:13:08.921119 899 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:08.927811 899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28c7dd643bb94654ad33f54912e02c59: Generated new TSK 0
I20260807 20:13:08.928023 899 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:13:08.965864 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:46377
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:09.048856 917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:09.049072 917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:09.049126 917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:09.050292 917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:09.050335 917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:09.050370 917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:13:09.051831 917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:46377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:09.052074 917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:09.052299 917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:09.057400 928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:09.057402 932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:09.057402 927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:09.057420 917 server_base.cc:1034] running on GCE node
I20260807 20:13:09.057988 917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:09.058239 917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:09.059386 917 hybrid_clock.cc:648] HybridClock initialized: now 1786133589059365 us; error 39 us; skew 500 ppm
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133589, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:09.062002 917 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:13:09.062481 917 webserver.cc:460] Webserver started at http://127.23.200.193:34537/ using document root <none> and password file <none>
I20260807 20:13:09.062690 917 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:09.062738 917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:09.062865 917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:09.063920 917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "5ca60f2b43ca498fa2edd79c1c3f9d28"
format_stamp: "Formatted at 2026-08-07 20:13:09 on dist-test-slave-2j7r"
server_key: "51dfa5daff6a8d1002c5cb72c337cafc"
server_key_iv: "5117295e684ec657239421a86d03d210"
server_key_version: "encryptionkey@0"
I20260807 20:13:09.064257 917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "5ca60f2b43ca498fa2edd79c1c3f9d28"
format_stamp: "Formatted at 2026-08-07 20:13:09 on dist-test-slave-2j7r"
server_key: "51dfa5daff6a8d1002c5cb72c337cafc"
server_key_iv: "5117295e684ec657239421a86d03d210"
server_key_version: "encryptionkey@0"
I20260807 20:13:09.065583 917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 20:13:09.066233 942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:09.066383 917 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:09.066474 917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "5ca60f2b43ca498fa2edd79c1c3f9d28"
format_stamp: "Formatted at 2026-08-07 20:13:09 on dist-test-slave-2j7r"
server_key: "51dfa5daff6a8d1002c5cb72c337cafc"
server_key_iv: "5117295e684ec657239421a86d03d210"
server_key_version: "encryptionkey@0"
I20260807 20:13:09.066532 917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:09.078083 917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:09.079864 917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:09.080004 917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:09.080222 917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:09.080530 917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:09.080564 917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:09.080595 917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:09.080618 917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:09.086146 917 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:36687
I20260807 20:13:09.086207 1132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:36687 every 8 connection(s)
I20260807 20:13:09.086522 917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:13:09.091727 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 917
I20260807 20:13:09.091843 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:13:09.092020 24355 external_mini_cluster.cc:1428] Setting key 7bf58ff0d540a73a28efe158e91de0d6
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133589, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:09.095748 904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:09.087340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:46369 (local address 127.23.200.254:46377)
0807 20:13:09.087424 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:09.087426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:09.087567 (+ 141us) server_negotiation.cc:407] Connection header received
0807 20:13:09.088537 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:09.088539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:09.088554 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:09.088607 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:09.089247 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.089744 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.090673 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.090873 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.090929 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:09.093112 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:09.093114 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:09.093114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:09.093123 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:09.094458 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:09.094826 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:09.094827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:09.094828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:09.094884 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:09.095106 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:09.095107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:09.095107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:09.095221 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:09.095277 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:09.095552 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 20:13:09.095655 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 20:13:09.095898 1136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:09.087271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:46377 (local address 127.23.200.193:46369)
0807 20:13:09.087445 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:09.087484 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:09.088441 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:09.088677 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:09.088681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:09.088964 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:09.089180 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:09.089184 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.089806 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.089807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:09.090611 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:09.090613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.090945 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.090946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:09.091042 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:09.091479 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:09.091488 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:09.092994 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:09.094535 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:09.094543 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:09.094545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:09.094760 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:09.094944 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:09.094945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:09.094945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:09.095050 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:09.095277 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:09.095284 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:09.095429 (+ 145us) client_negotiation.cc:769] Sending connection context
0807 20:13:09.095520 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 20:13:09.095677 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":71,"threads_started":1}
I20260807 20:13:09.096367 1133 heartbeater.cc:347] Connected to a master server at 127.23.200.254:46377
I20260807 20:13:09.096463 1133 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:09.096639 1133 heartbeater.cc:511] Master 127.23.200.254:46377 requested a full tablet report, sending...
I20260807 20:13:09.097267 813 ts_manager.cc:194] Registered new tserver with Master: 5ca60f2b43ca498fa2edd79c1c3f9d28 (127.23.200.193:36687)
I20260807 20:13:09.098183 813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:46369
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:13:09.118057 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:46377
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:09.201036 1139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:09.201313 1139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:09.201347 1139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:09.202598 1139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:09.202651 1139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:09.202688 1139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:13:09.204355 1139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:46377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:09.204638 1139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:09.204877 1139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:09.210484 1151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:09.210498 1150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:09.210484 1157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:09.210739 1139 server_base.cc:1034] running on GCE node
I20260807 20:13:09.211015 1139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:09.211287 1139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:09.212435 1139 hybrid_clock.cc:648] HybridClock initialized: now 1786133589212402 us; error 48 us; skew 500 ppm
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133589, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:09.215131 1139 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:13:09.215633 1139 webserver.cc:460] Webserver started at http://127.23.200.194:41077/ using document root <none> and password file <none>
I20260807 20:13:09.215871 1139 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:09.215919 1139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:09.216040 1139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:09.217126 1139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "87acea284b5747c282064748ebc205be"
format_stamp: "Formatted at 2026-08-07 20:13:09 on dist-test-slave-2j7r"
server_key: "48a1ddf829f5c86a31aef7801d8ce92c"
server_key_iv: "78c0a6eb0bab13941ef9ee831db0400e"
server_key_version: "encryptionkey@0"
I20260807 20:13:09.217480 1139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "87acea284b5747c282064748ebc205be"
format_stamp: "Formatted at 2026-08-07 20:13:09 on dist-test-slave-2j7r"
server_key: "48a1ddf829f5c86a31aef7801d8ce92c"
server_key_iv: "78c0a6eb0bab13941ef9ee831db0400e"
server_key_version: "encryptionkey@0"
I20260807 20:13:09.218772 1139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:09.219513 1171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:09.219676 1139 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:09.219769 1139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "87acea284b5747c282064748ebc205be"
format_stamp: "Formatted at 2026-08-07 20:13:09 on dist-test-slave-2j7r"
server_key: "48a1ddf829f5c86a31aef7801d8ce92c"
server_key_iv: "78c0a6eb0bab13941ef9ee831db0400e"
server_key_version: "encryptionkey@0"
I20260807 20:13:09.219830 1139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:09.229985 1139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:09.232275 1139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:09.232427 1139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:09.232661 1139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:09.232992 1139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:09.233027 1139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:09.233058 1139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:09.233152 1139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:09.239321 1139 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:40227
I20260807 20:13:09.239372 1362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:40227 every 8 connection(s)
I20260807 20:13:09.239718 1139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:13:09.243324 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 1139
I20260807 20:13:09.243465 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:13:09.243692 24355 external_mini_cluster.cc:1428] Setting key 628bf7d203dfe2401b84ddaa37a6c306
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133589, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:09.250159 904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:09.240502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:59451 (local address 127.23.200.254:46377)
0807 20:13:09.240565 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:09.240568 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:09.240785 (+ 217us) server_negotiation.cc:407] Connection header received
0807 20:13:09.241856 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:09.241857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:09.241869 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:09.241942 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:09.242604 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.243186 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.244582 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.244782 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.244834 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:09.247542 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:09.247543 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:09.247544 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:09.247553 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:09.248784 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:09.249211 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:09.249213 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:09.249213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:09.249270 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:09.249512 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:09.249513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:09.249513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:09.249626 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:09.249681 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:09.249965 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 20:13:09.250059 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:13:09.250514 1369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:09.240500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:46377 (local address 127.23.200.194:59451)
0807 20:13:09.240661 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:09.240701 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:09.241750 (+ 1049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:09.242009 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:09.242013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:09.242286 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:09.242498 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:09.242503 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.243285 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.243286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:09.244488 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:09.244492 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.244862 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.244863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:09.245035 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:09.245656 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:09.245667 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:09.247407 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:09.248866 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:09.248874 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:09.248876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:09.249148 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:09.249324 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:09.249324 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:09.249325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:09.249444 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:09.249681 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:09.249689 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:09.249833 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 20:13:09.249921 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 20:13:09.250171 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260807 20:13:09.250989 1363 heartbeater.cc:347] Connected to a master server at 127.23.200.254:46377
I20260807 20:13:09.251101 1363 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:09.251303 1363 heartbeater.cc:511] Master 127.23.200.254:46377 requested a full tablet report, sending...
I20260807 20:13:09.251698 813 ts_manager.cc:194] Registered new tserver with Master: 87acea284b5747c282064748ebc205be (127.23.200.194:40227)
I20260807 20:13:09.252089 813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:59451
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:13:09.272078 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:46377
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:09.355806 1375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:09.356019 1375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:09.356051 1375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:09.357236 1375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:09.357278 1375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:09.357314 1375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:13:09.358808 1375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:46377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:09.359055 1375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:09.359277 1375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:13:09.364588 1375 server_base.cc:1034] running on GCE node
W20260807 20:13:09.364526 1386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:09.364522 1384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:09.364548 1390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:09.364979 1375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:09.365302 1375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:09.366456 1375 hybrid_clock.cc:648] HybridClock initialized: now 1786133589366444 us; error 22 us; skew 500 ppm
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133589, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:09.369029 1375 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:13:09.369618 1375 webserver.cc:460] Webserver started at http://127.23.200.195:35843/ using document root <none> and password file <none>
I20260807 20:13:09.369863 1375 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:09.369915 1375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:09.370044 1375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:09.371109 1375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "ae205d7dd8ae4c92bc709b50edf9fa65"
format_stamp: "Formatted at 2026-08-07 20:13:09 on dist-test-slave-2j7r"
server_key: "9453e60a29ec42f4c024f9fa24d50ef7"
server_key_iv: "6895c1534f15e739b7c39d0906429990"
server_key_version: "encryptionkey@0"
I20260807 20:13:09.371460 1375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "ae205d7dd8ae4c92bc709b50edf9fa65"
format_stamp: "Formatted at 2026-08-07 20:13:09 on dist-test-slave-2j7r"
server_key: "9453e60a29ec42f4c024f9fa24d50ef7"
server_key_iv: "6895c1534f15e739b7c39d0906429990"
server_key_version: "encryptionkey@0"
I20260807 20:13:09.372779 1375 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:13:09.373571 1404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:09.373708 1375 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:09.373798 1375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "ae205d7dd8ae4c92bc709b50edf9fa65"
format_stamp: "Formatted at 2026-08-07 20:13:09 on dist-test-slave-2j7r"
server_key: "9453e60a29ec42f4c024f9fa24d50ef7"
server_key_iv: "6895c1534f15e739b7c39d0906429990"
server_key_version: "encryptionkey@0"
I20260807 20:13:09.373860 1375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:09.387746 1375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:09.389649 1375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:09.389791 1375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:09.390017 1375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:09.390326 1375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:09.390360 1375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:09.390400 1375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:09.390424 1375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:09.396144 1375 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:35481
I20260807 20:13:09.396198 1589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:35481 every 8 connection(s)
I20260807 20:13:09.396524 1375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:13:09.397903 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 1375
I20260807 20:13:09.398011 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:13:09.398237 24355 external_mini_cluster.cc:1428] Setting key be79cc2003c668deea0ed3d00eff24dd
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133589, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:09.406622 904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:09.397293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:37627 (local address 127.23.200.254:46377)
0807 20:13:09.397341 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:09.397343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:09.397564 (+ 221us) server_negotiation.cc:407] Connection header received
0807 20:13:09.398742 (+ 1178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:09.398743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:09.398756 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:09.398821 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:09.399583 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.400112 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.401177 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.401369 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.401415 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:09.403698 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:09.403700 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:09.403700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:09.403708 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:09.405133 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:09.405594 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:09.405595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:09.405596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:09.405676 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:09.405930 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:09.405930 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:09.405930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:09.406067 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:09.406126 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:09.406448 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 20:13:09.406529 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:13:09.406782 1592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:09.397272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:46377 (local address 127.23.200.195:37627)
0807 20:13:09.397436 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:09.397480 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:09.398641 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:09.398909 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:09.398913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:09.399260 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:09.399503 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:09.399509 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.400181 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.400182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:09.401056 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:09.401058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.401470 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.401471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:09.401586 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:09.402025 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:09.402034 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:09.403588 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:09.405205 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:09.405213 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:09.405216 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:09.405455 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:09.405738 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:09.405739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:09.405739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:09.405856 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:09.406127 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:09.406135 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:09.406321 (+ 186us) client_negotiation.cc:769] Sending connection context
0807 20:13:09.406428 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 20:13:09.406571 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260807 20:13:09.407261 1590 heartbeater.cc:347] Connected to a master server at 127.23.200.254:46377
I20260807 20:13:09.407361 1590 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:09.407593 1590 heartbeater.cc:511] Master 127.23.200.254:46377 requested a full tablet report, sending...
I20260807 20:13:09.407979 813 ts_manager.cc:194] Registered new tserver with Master: ae205d7dd8ae4c92bc709b50edf9fa65 (127.23.200.195:35481)
I20260807 20:13:09.408316 813 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:37627
I20260807 20:13:09.410005 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/test-admin.keytab.
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:09.426191 24355 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 20:13:09.427947 1604 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:09.427154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33738 (local address 127.23.200.193:36687)
0807 20:13:09.427353 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:09.427358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:09.427368 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:13:09.427395 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:09.427397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:09.427412 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:09.427527 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:09.427861 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33738: BlockingRecv error: recv got EOF from 127.0.0.1:33738 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":118,"threads_started":1}
W20260807 20:13:09.428162 24355 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/krb5cc)
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:09.429212 24355 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:09.430075 24355 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.193@KRBTEST.COM
I20260807 20:13:09.437472 1604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:09.430625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33740 (local address 127.23.200.193:36687)
0807 20:13:09.430686 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:09.430688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:09.430696 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:13:09.430745 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:09.430746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:09.430757 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:09.430802 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:09.431164 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.431907 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.432619 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.432956 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.433011 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:09.434993 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:09.435002 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:09.435002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:09.435009 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:09.436465 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:09.436741 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:09.436743 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:09.436744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:09.436792 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:09.436945 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:09.436946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:09.436946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:09.437071 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:09.437205 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:09.437260 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 20:13:09.437324 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 20:13:09.439282 1604 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:09.438722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33746 (local address 127.23.200.193:36687)
0807 20:13:09.438779 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:09.438780 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:09.438788 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:13:09.438833 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:09.438834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:09.438844 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:09.438883 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:09.439234 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33746: BlockingRecv error: recv got EOF from 127.0.0.1:33746 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 20:13:09.439363 24355 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/krb5cc' of type FILE
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:09.440384 24355 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:09.441241 24355 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.193@KRBTEST.COM
I20260807 20:13:09.448037 1604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:09.441754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33748 (local address 127.23.200.193:36687)
0807 20:13:09.441793 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:09.441795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:09.441826 (+ 31us) server_negotiation.cc:407] Connection header received
0807 20:13:09.441875 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:09.441875 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:09.441885 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:09.441929 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:09.442360 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.442854 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.443557 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.443752 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.443797 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:09.445831 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:09.445833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:09.445833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:09.445841 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:09.447068 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:09.447429 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:09.447429 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:09.447430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:09.447479 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:09.447669 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:09.447669 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:09.447669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:09.447760 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:09.447832 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:09.447909 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 20:13:09.447971 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 20:13:09.449538 1604 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:09.449024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33764 (local address 127.23.200.193:36687)
0807 20:13:09.449066 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:09.449068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:09.449134 (+ 66us) server_negotiation.cc:407] Connection header received
0807 20:13:09.449197 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:09.449197 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:09.449205 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:09.449239 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:09.449489 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33764: BlockingRecv error: recv got EOF from 127.0.0.1:33764 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 20:13:09.449646 24355 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133562076833-24355-0/krb5kdc/krb5cc' of type FILE
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:09.450690 24355 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:09.451665 24355 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.193@KRBTEST.COM
I20260807 20:13:09.458673 1604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:09.452236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33774 (local address 127.23.200.193:36687)
0807 20:13:09.452271 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:09.452272 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:09.452365 (+ 93us) server_negotiation.cc:407] Connection header received
0807 20:13:09.452433 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:09.452434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:09.452443 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:09.452486 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:09.452865 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.453531 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.454207 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:09.454426 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:09.454490 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:09.456369 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:09.456370 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:09.456371 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:09.456378 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:09.457777 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:09.458086 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:09.458087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:09.458087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:09.458131 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:09.458291 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:09.458291 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:09.458291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:09.458380 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:09.458451 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:09.458528 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 20:13:09.458586 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 20:13:09.459295 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 917
I20260807 20:13:09.476542 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 1139
I20260807 20:13:09.488885 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 1375
I20260807 20:13:09.496769 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 764
2026-08-07T20:13:09Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2057 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[1624](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[1624](info): set up 2 sockets
Aug 07 20:13:09 dist-test-slave-2j7r krb5kdc[1624](info): commencing operation
krb5kdc: starting...
W20260807 20:13:10.663915 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:13:10 dist-test-slave-2j7r krb5kdc[1624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:13:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:10Z Disabled control of system clock
WARNING: no policy specified for oryx/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.254.keytab.
Entry for principal oryx/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.254.keytab.
I20260807 20:13:10.791790 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:37725
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.254.keytab
--principal=oryx/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:33189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:37725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:10.877990 1643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:10.878199 1643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:10.878221 1643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:10.879487 1643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:10.879539 1643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:10.879557 1643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:10.879585 1643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:10.879599 1643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:13:10.881198 1643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:33189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:37725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.254.keytab
--principal=oryx/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:37725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:10.881456 1643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:10.881678 1643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:10.886711 1654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:10.886711 1648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:10.886922 1643 server_base.cc:1034] running on GCE node
W20260807 20:13:10.886706 1650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:10.887305 1643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:10.887591 1643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:10.888728 1643 hybrid_clock.cc:648] HybridClock initialized: now 1786133590888716 us; error 31 us; skew 500 ppm
Aug 07 20:13:10 dist-test-slave-2j7r krb5kdc[1624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133590, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:10.895341 1643 init.cc:377] Logged in from keytab as oryx/127.23.200.254@KRBTEST.COM (short username oryx)
I20260807 20:13:10.895861 1643 webserver.cc:460] Webserver started at http://127.23.200.254:44391/ using document root <none> and password file <none>
I20260807 20:13:10.896083 1643 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:10.896126 1643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:10.896222 1643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:10.897316 1643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "9859649d8d244f6983ca11af92fdb052"
format_stamp: "Formatted at 2026-08-07 20:13:10 on dist-test-slave-2j7r"
server_key: "041c2dbbf81eec57eb561232a6402450"
server_key_iv: "13ab3b246a2b7c1f70771feeeea6d6c7"
server_key_version: "encryptionkey@0"
I20260807 20:13:10.897634 1643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "9859649d8d244f6983ca11af92fdb052"
format_stamp: "Formatted at 2026-08-07 20:13:10 on dist-test-slave-2j7r"
server_key: "041c2dbbf81eec57eb561232a6402450"
server_key_iv: "13ab3b246a2b7c1f70771feeeea6d6c7"
server_key_version: "encryptionkey@0"
I20260807 20:13:10.898875 1643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 20:13:10.899639 1664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:10.899834 1643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:13:10.899923 1643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "9859649d8d244f6983ca11af92fdb052"
format_stamp: "Formatted at 2026-08-07 20:13:10 on dist-test-slave-2j7r"
server_key: "041c2dbbf81eec57eb561232a6402450"
server_key_iv: "13ab3b246a2b7c1f70771feeeea6d6c7"
server_key_version: "encryptionkey@0"
I20260807 20:13:10.899981 1643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:10.919243 1643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:10.921489 1643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:10.921604 1643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:10.925506 1643 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:37725
I20260807 20:13:10.925550 1758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:37725 every 8 connection(s)
I20260807 20:13:10.925896 1643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:13:10.927003 1760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:10.927477 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 1643
I20260807 20:13:10.927578 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:13:10.927769 24355 external_mini_cluster.cc:1428] Setting key 2e360791d234c67dc17c38188c6a0e7a
I20260807 20:13:10.930006 1760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052: Bootstrap starting.
I20260807 20:13:10.930730 1760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:10.931041 1760 log.cc:824] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:10.931732 1760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052: No bootstrap required, opened a new log
I20260807 20:13:10.933848 1760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } }
I20260807 20:13:10.933990 1760 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:10.934015 1760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9859649d8d244f6983ca11af92fdb052, State: Initialized, Role: FOLLOWER
I20260807 20:13:10.934149 1760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } }
I20260807 20:13:10.934249 1760 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:13:10.934286 1760 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:13:10.934333 1760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [term 0 FOLLOWER]: Advancing to term 1
Aug 07 20:13:10 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM
I20260807 20:13:10.934993 1760 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } }
I20260807 20:13:10.935096 1760 leader_election.cc:302] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9859649d8d244f6983ca11af92fdb052; no voters:
I20260807 20:13:10.935272 1760 leader_election.cc:288] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:13:10.935320 1774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:10.935457 1774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [term 1 LEADER]: Becoming Leader. State: Replica: 9859649d8d244f6983ca11af92fdb052, State: Running, Role: LEADER
I20260807 20:13:10.935557 1760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:10.935549 1774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } }
I20260807 20:13:10.936013 1776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9859649d8d244f6983ca11af92fdb052. Latest consensus state: current_term: 1 leader_uuid: "9859649d8d244f6983ca11af92fdb052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } }
I20260807 20:13:10.936096 1776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:10.936247 1775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9859649d8d244f6983ca11af92fdb052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } }
I20260807 20:13:10.936319 1775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: This master's current role is: LEADER
W20260807 20:13:10.937422 1796 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:13:10.937477 1796 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:13:10.937530 1785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:13:10.937704 1785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:10.938992 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:10.928338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53388 (local address 127.23.200.254:37725)
0807 20:13:10.928526 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:10.928536 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:10.928579 (+ 43us) server_negotiation.cc:407] Connection header received
0807 20:13:10.929125 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:10.929135 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:10.929214 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:10.929373 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:10.929968 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:10.931297 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:10.932057 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:10.932650 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:10.932763 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:10.935138 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:10.935142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:10.935143 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:10.935156 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:10.937220 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:10.937651 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:10.937655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:10.937656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:10.937764 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:10.937971 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:10.937972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:10.937972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:10.938269 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:10.938424 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:10.938639 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 20:13:10.938779 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260807 20:13:10.941463 1785 catalog_manager.cc:1347] Generated new cluster ID: 09fc8a617b444902b2c24ba4c910146c
I20260807 20:13:10.941512 1785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:13:10.955824 1785 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:13:10.956532 1785 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:10.964514 1785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052: Generated new TSK 0
I20260807 20:13:10.964711 1785 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.193.keytab.
Entry for principal oryx/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.193.keytab.
I20260807 20:13:10.999842 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37725
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.193.keytab
--principal=oryx/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:33189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:11.099323 1801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:11.099529 1801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:11.099552 1801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:11.100623 1801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:11.100656 1801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:11.100682 1801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:13:11.102119 1801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:33189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.193.keytab
--principal=oryx/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:11.102309 1801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:11.102495 1801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:11.107528 1807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:11.107523 1808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:11.107523 1812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:11.107769 1801 server_base.cc:1034] running on GCE node
I20260807 20:13:11.107959 1801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:11.108162 1801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:11.109297 1801 hybrid_clock.cc:648] HybridClock initialized: now 1786133591109284 us; error 31 us; skew 500 ppm
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133591, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:11.111750 1801 init.cc:377] Logged in from keytab as oryx/127.23.200.193@KRBTEST.COM (short username oryx)
I20260807 20:13:11.112224 1801 webserver.cc:460] Webserver started at http://127.23.200.193:36801/ using document root <none> and password file <none>
I20260807 20:13:11.112437 1801 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:11.112486 1801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:11.112615 1801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:11.113740 1801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "c0b79210210043cd8434b1c3e065f28c"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "a6cfadad3ec028f0b4711ca80e561272"
server_key_iv: "d31bb957e77909de33f90901cda74ecc"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.114114 1801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "c0b79210210043cd8434b1c3e065f28c"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "a6cfadad3ec028f0b4711ca80e561272"
server_key_iv: "d31bb957e77909de33f90901cda74ecc"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.115458 1801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:13:11.116117 1825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.116257 1801 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:11.116343 1801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "c0b79210210043cd8434b1c3e065f28c"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "a6cfadad3ec028f0b4711ca80e561272"
server_key_iv: "d31bb957e77909de33f90901cda74ecc"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.116400 1801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:11.141242 1801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:11.143309 1801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:11.143453 1801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:11.143704 1801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:11.144021 1801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:11.144054 1801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.144084 1801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:11.144102 1801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.149927 1801 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:44069
I20260807 20:13:11.149977 2010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:44069 every 8 connection(s)
I20260807 20:13:11.150296 1801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133591, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM
I20260807 20:13:11.156801 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 1801
I20260807 20:13:11.156903 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:13:11.157192 24355 external_mini_cluster.cc:1428] Setting key 8ce5878714ea02da9e5b3682247c3858
I20260807 20:13:11.160686 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.150989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:35569 (local address 127.23.200.254:37725)
0807 20:13:11.151054 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.151056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.151240 (+ 184us) server_negotiation.cc:407] Connection header received
0807 20:13:11.151705 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.151707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.151716 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.151766 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.152674 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.153202 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.154155 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.154354 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.154398 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.156364 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.156366 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.156366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.156374 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.157864 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.159737 (+ 1873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.159740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.159740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.159805 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.160035 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.160036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.160036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.160152 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.160209 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.160486 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.160592 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 20:13:11.160857 2014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.150947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.193:35569)
0807 20:13:11.151118 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.151161 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.151633 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.151828 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.151831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.152398 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.152605 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.152609 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.153262 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.153263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.154092 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.154094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.154408 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.154408 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.154505 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.154973 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.154982 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.156275 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.159448 (+ 3173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.159454 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.159456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.159662 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.159864 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.159865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.159865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.159980 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.160205 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.160213 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.160360 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.160456 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.160629 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
I20260807 20:13:11.161368 2012 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37725
I20260807 20:13:11.161470 2012 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:11.161648 2012 heartbeater.cc:511] Master 127.23.200.254:37725 requested a full tablet report, sending...
I20260807 20:13:11.162164 1692 ts_manager.cc:194] Registered new tserver with Master: c0b79210210043cd8434b1c3e065f28c (127.23.200.193:44069)
I20260807 20:13:11.162770 1692 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:35569
WARNING: no policy specified for oryx/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.194.keytab.
Entry for principal oryx/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.194.keytab.
I20260807 20:13:11.184150 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37725
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.194.keytab
--principal=oryx/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:33189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:11.266227 2018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:11.266425 2018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:11.266446 2018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:11.267514 2018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:11.267546 2018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:11.267570 2018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:13:11.268985 2018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:33189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.194.keytab
--principal=oryx/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:11.269210 2018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:11.269398 2018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:11.274245 2028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:11.274243 2026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:11.274296 2018 server_base.cc:1034] running on GCE node
W20260807 20:13:11.274277 2030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:11.274672 2018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:11.274883 2018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:11.276072 2018 hybrid_clock.cc:648] HybridClock initialized: now 1786133591276057 us; error 28 us; skew 500 ppm
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133591, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:11.278594 2018 init.cc:377] Logged in from keytab as oryx/127.23.200.194@KRBTEST.COM (short username oryx)
I20260807 20:13:11.279016 2018 webserver.cc:460] Webserver started at http://127.23.200.194:44183/ using document root <none> and password file <none>
I20260807 20:13:11.279186 2018 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:11.279232 2018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:11.279325 2018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:11.280288 2018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "8ba1c663c66d4941ba2bc5fb0f482053"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "5c719eb02d7ee29d07cfebbc5452f175"
server_key_iv: "e80c660b3046674eba274f7eb2f2ddae"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.280601 2018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "8ba1c663c66d4941ba2bc5fb0f482053"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "5c719eb02d7ee29d07cfebbc5452f175"
server_key_iv: "e80c660b3046674eba274f7eb2f2ddae"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.281876 2018 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:13:11.282618 2039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.282815 2018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:13:11.282902 2018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "8ba1c663c66d4941ba2bc5fb0f482053"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "5c719eb02d7ee29d07cfebbc5452f175"
server_key_iv: "e80c660b3046674eba274f7eb2f2ddae"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.282977 2018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:11.310660 2018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:11.312403 2018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:11.312551 2018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:11.312770 2018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:11.313068 2018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:11.313143 2018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.313175 2018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:11.313199 2018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.318853 2018 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:46497
I20260807 20:13:11.318903 2219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:46497 every 8 connection(s)
I20260807 20:13:11.319228 2018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:13:11.319458 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 2018
I20260807 20:13:11.319561 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:13:11.319809 24355 external_mini_cluster.cc:1428] Setting key 765bb49a0754c8b72de5c1967e78db5f
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133591, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.23.200.195@KRBTEST.COM; defaulting to no policy
I20260807 20:13:11.329378 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.320040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:34951 (local address 127.23.200.254:37725)
0807 20:13:11.320096 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.320098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.320361 (+ 263us) server_negotiation.cc:407] Connection header received
0807 20:13:11.320964 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.320966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.320977 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.321039 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.322007 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.322635 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.323942 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.324206 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.324283 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.326695 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.326696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.326697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.326704 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.327979 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.328443 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.328444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.328444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.328492 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.328744 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.328745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.328745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.328841 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.328886 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.329234 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.329307 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 20:13:11.329617 2226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.320041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.194:34951)
0807 20:13:11.320209 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.320258 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.320869 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.321134 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.321138 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.321720 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.321922 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.321926 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.322727 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.322728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.323843 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.323845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.324297 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.324298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.324407 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.324922 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.324934 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.326596 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.328063 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.328071 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.328074 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.328367 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.328547 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.328548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.328548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.328685 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.328892 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.328898 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.329070 (+ 172us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.329217 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.329399 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260807 20:13:11.330049 2221 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37725
I20260807 20:13:11.330157 2221 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:11.330343 2221 heartbeater.cc:511] Master 127.23.200.254:37725 requested a full tablet report, sending...
I20260807 20:13:11.330705 1692 ts_manager.cc:194] Registered new tserver with Master: 8ba1c663c66d4941ba2bc5fb0f482053 (127.23.200.194:46497)
I20260807 20:13:11.331045 1692 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:34951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.195.keytab.
Entry for principal oryx/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.195.keytab.
I20260807 20:13:11.345811 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37725
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.195.keytab
--principal=oryx/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:33189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:11.427918 2231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:11.428143 2231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:11.428179 2231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:11.429387 2231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:11.429433 2231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:11.429469 2231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:13:11.430958 2231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:33189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.195.keytab
--principal=oryx/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:11.431216 2231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:11.431440 2231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:11.436800 2241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:11.436779 2242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:11.436946 2231 server_base.cc:1034] running on GCE node
W20260807 20:13:11.436792 2244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:11.437363 2231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:11.437638 2231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:11.438796 2231 hybrid_clock.cc:648] HybridClock initialized: now 1786133591438771 us; error 36 us; skew 500 ppm
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133591, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:11.441339 2231 init.cc:377] Logged in from keytab as oryx/127.23.200.195@KRBTEST.COM (short username oryx)
I20260807 20:13:11.441780 2231 webserver.cc:460] Webserver started at http://127.23.200.195:34073/ using document root <none> and password file <none>
I20260807 20:13:11.442008 2231 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:11.442056 2231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:11.442186 2231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:11.443210 2231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "6f7ecf2c69394d2eb92172f67b98d8aa"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "8c44bdc44cd74fc78b3cb636ea6f7b48"
server_key_iv: "e69120de494d1ebb2719155fe277eb23"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.443545 2231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "6f7ecf2c69394d2eb92172f67b98d8aa"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "8c44bdc44cd74fc78b3cb636ea6f7b48"
server_key_iv: "e69120de494d1ebb2719155fe277eb23"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.444778 2231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:13:11.445484 2253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.445614 2231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.445700 2231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "6f7ecf2c69394d2eb92172f67b98d8aa"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "8c44bdc44cd74fc78b3cb636ea6f7b48"
server_key_iv: "e69120de494d1ebb2719155fe277eb23"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.445758 2231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:11.460893 2231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:11.463102 2231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:11.463248 2231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:11.463482 2231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:11.463797 2231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:11.463831 2231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.463861 2231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:11.463884 2231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.469798 2231 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:43335
I20260807 20:13:11.469849 2441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:43335 every 8 connection(s)
I20260807 20:13:11.470161 2231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:13:11.470726 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 2231
I20260807 20:13:11.470818 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:13:11.471015 24355 external_mini_cluster.cc:1428] Setting key a66e97ee66fd65eda1169c1cc0455162
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133591, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM
I20260807 20:13:11.479261 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.470844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:48825 (local address 127.23.200.254:37725)
0807 20:13:11.470895 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.470897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.471206 (+ 309us) server_negotiation.cc:407] Connection header received
0807 20:13:11.471884 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.471885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.471899 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.471961 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.472873 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.473429 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.474448 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.474642 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.474701 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.476777 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.476779 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.476779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.476787 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.478124 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.478535 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.478535 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.478536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.478583 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.478759 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.478759 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.478760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.478854 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.478897 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.479113 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.479173 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:13:11.479478 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.470822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.195:48825)
0807 20:13:11.471055 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.471107 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.471787 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.472024 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.472027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.472589 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.472782 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.472787 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.473509 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.473510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.474330 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.474333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.474707 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.474707 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.474802 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.475245 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.475253 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.476685 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.478193 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.478200 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.478202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.478477 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.478625 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.478626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.478626 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.478719 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.478896 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.478899 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.479014 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.479093 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.479212 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":85,"threads_started":1}
I20260807 20:13:11.479969 2443 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37725
I20260807 20:13:11.480062 2443 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:11.480242 2443 heartbeater.cc:511] Master 127.23.200.254:37725 requested a full tablet report, sending...
I20260807 20:13:11.480628 1692 ts_manager.cc:194] Registered new tserver with Master: 6f7ecf2c69394d2eb92172f67b98d8aa (127.23.200.195:43335)
I20260807 20:13:11.480979 1692 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:48825
I20260807 20:13:11.482626 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.23.200.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.200.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.200.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.253.keytab.
Entry for principal oryx/127.23.200.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.253.keytab.
I20260807 20:13:11.511746 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.253:42419
--webserver_interface=127.23.200.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.253.keytab
--principal=oryx/127.23.200.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:33189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:37725,127.23.200.253:42419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:11.594472 2448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:11.594684 2448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:11.594707 2448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:11.595851 2448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:11.595894 2448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:11.595907 2448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:11.595918 2448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:11.595928 2448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:13:11.597447 2448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:33189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:37725,127.23.200.253:42419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.253.keytab
--principal=oryx/127.23.200.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.253:42419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.200.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
W20260807 20:13:11.597558 2448 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.23.200.254:37725,127.23.200.253:42419'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 20:13:11.597708 2448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:11.597922 2448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:11.602813 2455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:11.602820 2456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:11.602835 2448 server_base.cc:1034] running on GCE node
W20260807 20:13:11.602819 2458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:11.603186 2448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:11.603505 2448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:11.604660 2448 hybrid_clock.cc:648] HybridClock initialized: now 1786133591604641 us; error 32 us; skew 500 ppm
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133591, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:11.607201 2448 init.cc:377] Logged in from keytab as oryx/127.23.200.253@KRBTEST.COM (short username oryx)
I20260807 20:13:11.607640 2448 webserver.cc:460] Webserver started at http://127.23.200.253:32909/ using document root <none> and password file <none>
I20260807 20:13:11.607883 2448 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:11.607930 2448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:11.608040 2448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:11.609052 2448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/data/instance:
uuid: "d80a1bea047e480394df4430b235c8aa"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "ddd7a8ffbd63776b2cfaf0ab3666cfe1"
server_key_iv: "cecc27221a42260d8bece661e1738fc4"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.609426 2448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/wal/instance:
uuid: "d80a1bea047e480394df4430b235c8aa"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "ddd7a8ffbd63776b2cfaf0ab3666cfe1"
server_key_iv: "cecc27221a42260d8bece661e1738fc4"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.610690 2448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:13:11.611377 2472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.611536 2448 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:13:11.611622 2448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/wal
uuid: "d80a1bea047e480394df4430b235c8aa"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "ddd7a8ffbd63776b2cfaf0ab3666cfe1"
server_key_iv: "cecc27221a42260d8bece661e1738fc4"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.611680 2448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:11.624120 2448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:11.626205 2448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:11.626338 2448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:11.630182 2448 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.253:42419
I20260807 20:13:11.630218 2553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.253:42419 every 8 connection(s)
I20260807 20:13:11.630543 2448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/data/info.pb
I20260807 20:13:11.631513 2555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:11.633018 2555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:13:11.637295 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 2448
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133591, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.253@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM
I20260807 20:13:11.641726 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.633762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53400 (local address 127.23.200.254:37725)
0807 20:13:11.633838 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.633840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.634012 (+ 172us) server_negotiation.cc:407] Connection header received
0807 20:13:11.634554 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.634555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.634569 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.634620 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.635218 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.635755 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.636769 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.636956 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.637010 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.639200 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.639202 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.639202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.639209 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.640416 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.640783 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.640784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.640785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.640836 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.641052 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.641052 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.641053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.641239 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.641301 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.641552 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.641636 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 20:13:11.641867 2561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.633694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:53400)
0807 20:13:11.633889 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.633925 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.634453 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.634686 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.634689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.634945 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.635132 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.635136 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.635836 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.635837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.636698 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.636700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.637023 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.637024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.637195 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.637708 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.637727 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.639122 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.640490 (+ 1368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.640497 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.640499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.640705 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.640890 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.640891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.640891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.640996 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.641299 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.641306 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.641438 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.641523 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.641660 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
I20260807 20:13:11.642481 2555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.200.253@KRBTEST.COM
I20260807 20:13:11.645628 2565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.637783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55530 (local address 127.23.200.253:42419)
0807 20:13:11.637959 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.637964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.637989 (+ 25us) server_negotiation.cc:407] Connection header received
0807 20:13:11.638029 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.638031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.638052 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.638115 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.638876 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.639649 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.640492 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.640785 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.640841 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.643180 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.643188 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.643188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.643195 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.644568 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.644872 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.644875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.644876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.644951 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.645130 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.645130 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.645131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.645261 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.645425 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.645468 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.645536 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":111,"threads_started":1}
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133591, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.253@KRBTEST.COM for oryx/127.23.200.253@KRBTEST.COM
I20260807 20:13:11.648739 2561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.642706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:42419 (local address 127.0.0.1:55542)
0807 20:13:11.642749 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.642759 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.642796 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.642956 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.642957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.643083 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.643120 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.643121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.643701 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.643702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.644159 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.644160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.644479 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.644480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.644540 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.644864 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.644868 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.646000 (+ 1132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.647886 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.647887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.647888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.648042 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.648199 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.648200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.648200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.648246 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.648527 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.648528 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.648564 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.648611 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.648681 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 20:13:11.648846 2567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.642736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55542 (local address 127.23.200.253:42419)
0807 20:13:11.642867 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.642868 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.642874 (+ 6us) server_negotiation.cc:407] Connection header received
0807 20:13:11.642891 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.642891 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.642899 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.642954 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.643180 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.643641 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.644221 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.644425 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.644475 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.646077 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.646078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.646078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.646083 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.647803 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.648104 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.648104 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.648105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.648154 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.648315 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.648315 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.648316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.648472 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.648523 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.648637 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.648728 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260807 20:13:11.650048 2555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: Bootstrap starting.
I20260807 20:13:11.650642 2555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:11.650930 2555 log.cc:824] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:11.651643 2555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: No bootstrap required, opened a new log
I20260807 20:13:11.652974 2555 raft_consensus.cc:348] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } }
I20260807 20:13:11.653110 2555 raft_consensus.cc:374] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:11.653154 2555 raft_consensus.cc:729] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d80a1bea047e480394df4430b235c8aa, State: Initialized, Role: FOLLOWER
I20260807 20:13:11.653271 2555 consensus_queue.cc:260] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } }
I20260807 20:13:11.653558 2555 sys_catalog.cc:564] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:11.653573 2570 sys_catalog.cc:455] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } } }
I20260807 20:13:11.653676 2570 sys_catalog.cc:458] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: This master's current role is: FOLLOWER
W20260807 20:13:11.655158 2586 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:13:11.655212 2586 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:13:11.656812 2448 master_runner.cc:428] Detected that this master d80a1bea047e480394df4430b235c8aa is joining an existing cluster
I20260807 20:13:11.656883 2448 master_runner.cc:432] Initiating AddMaster RPC to add 127.23.200.253:42419
I20260807 20:13:11.664578 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.658072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53412 (local address 127.23.200.254:37725)
0807 20:13:11.658113 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.658115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.658155 (+ 40us) server_negotiation.cc:407] Connection header received
0807 20:13:11.658217 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.658218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.658229 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.658285 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.658805 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.659525 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.660595 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.660785 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.660851 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.661940 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.661941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.661941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.661948 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.663371 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.663856 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.663857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.663857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.663918 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.664097 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.664098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.664098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.664204 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.664260 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.664387 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.664481 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:13:11.664597 2597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.657892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:53412)
0807 20:13:11.658058 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.658068 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.658164 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.658334 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.658336 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.658609 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.658690 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.658693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.659603 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.659604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.660507 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.660510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.660886 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.660888 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.660959 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.661445 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.661449 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.661836 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.663601 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.663602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.663602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.663790 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.663978 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.663978 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.663979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.664034 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.664255 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.664256 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.664298 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.664357 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.664496 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
I20260807 20:13:11.665714 2567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.657948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55556 (local address 127.23.200.253:42419)
0807 20:13:11.657987 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.657989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.658209 (+ 220us) server_negotiation.cc:407] Connection header received
0807 20:13:11.658288 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.658288 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.658297 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.658335 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.658766 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.659525 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.661535 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.661815 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.661883 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.662796 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.662797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.662798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.662803 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.664277 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.664876 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.664877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.664877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.664932 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.665123 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.665124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.665124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.665237 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.665276 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.665462 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.665601 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":19}
I20260807 20:13:11.665632 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.657953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:42419 (local address 127.0.0.1:55556)
0807 20:13:11.658158 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.658166 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.658234 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.658338 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.658339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.658618 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.658694 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.658696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.660604 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.660605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.661444 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.661448 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.661891 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.661892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.661974 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.662328 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.662332 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.662707 (+ 375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.664372 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.664373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.664373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.664615 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.664992 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.664993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.664993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.665042 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.665314 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.665314 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.665369 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.665432 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.665536 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":122,"threads_started":1}
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133590, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.254@KRBTEST.COM for oryx/127.23.200.253@KRBTEST.COM
I20260807 20:13:11.673910 2567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.666893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55568 (local address 127.23.200.253:42419)
0807 20:13:11.666933 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.666934 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.667079 (+ 145us) server_negotiation.cc:407] Connection header received
0807 20:13:11.667165 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.667166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.667173 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.667214 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.667813 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.668248 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.669326 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.669504 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.669554 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.671628 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.671629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.671630 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.671636 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.672807 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.673266 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.673267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.673267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.673314 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.673487 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.673488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.673488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.673580 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.673624 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.673792 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.673848 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:13:11.674006 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.666829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:42419 (local address 127.0.0.1:55568)
0807 20:13:11.666958 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.667010 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.667095 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.667221 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.667224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.667672 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.667732 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.667734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.668316 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.668317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.669257 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.669260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.669569 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.669570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.669655 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.670026 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.670031 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.671548 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.672876 (+ 1328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.672884 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.672887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.673201 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.673371 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.673372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.673372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.673434 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.673627 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.673634 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.673690 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.673777 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.673907 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
I20260807 20:13:11.674610 1692 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.23.200.253:42419: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d80a1bea047e480394df4430b235c8aa"
member_type: NON_VOTER
last_known_addr {
host: "127.23.200.253"
port: 42419
}
attrs {
promote: true
}
}
dest_uuid: "9859649d8d244f6983ca11af92fdb052"
cas_config_opid_index: -1
I20260807 20:13:11.674921 1692 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: NON_VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: true } }
I20260807 20:13:11.675376 2604 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d80a1bea047e480394df4430b235c8aa (127.23.200.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: NON_VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: true } } }
I20260807 20:13:11.675805 2515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:11.676280 2604 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.23.200.253:42419
I20260807 20:13:11.676362 2604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9859649d8d244f6983ca11af92fdb052" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: NON_VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: true } } }
I20260807 20:13:11.676421 2604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:11.676690 2448 master.cc:537] Master@127.23.200.253:42419 shutting down...
I20260807 20:13:11.677160 2515 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 FOLLOWER]: Refusing update from remote peer 9859649d8d244f6983ca11af92fdb052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 20:13:11.677524 2602 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: NON_VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 20:13:11.678321 1670 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 20:13:11.678710 1670 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 -> Peer d80a1bea047e480394df4430b235c8aa (127.23.200.253:42419): Couldn't send request to peer d80a1bea047e480394df4430b235c8aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 20:13:11.680038 2448 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:11.680141 2448 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:11.680205 2448 tablet_replica.cc:330] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: stopping tablet replica
I20260807 20:13:11.693009 2448 master.cc:552] Master@127.23.200.253:42419 shutdown complete.
I20260807 20:13:11.693070 2448 master_runner.cc:305] Clearing existing system tablet
I20260807 20:13:11.693238 2448 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:11.693298 2448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:11.694010 2448 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:11.694612 2612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.694747 2448 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:11.694808 2448 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:11.694831 2448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/wal
uuid: "d80a1bea047e480394df4430b235c8aa"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "ddd7a8ffbd63776b2cfaf0ab3666cfe1"
server_key_iv: "cecc27221a42260d8bece661e1738fc4"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.695739 2448 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:11.697243 2448 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 20:13:11.697324 2448 log.cc:1196] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 20:13:11.697521 2448 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: Deleting consensus metadata
I20260807 20:13:11.697736 2448 master_runner.cc:315] Copying system tablet from 127.23.200.254:37725
I20260807 20:13:11.698350 2448 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: tablet copy: Beginning tablet copy session from remote peer at address 127.23.200.254:37725
I20260807 20:13:11.704473 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.698709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53414 (local address 127.23.200.254:37725)
0807 20:13:11.698756 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.698758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.698799 (+ 41us) server_negotiation.cc:407] Connection header received
0807 20:13:11.698877 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.698878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.698888 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.698931 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.699259 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.699748 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.700763 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.700955 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.701002 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.702028 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.702029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.702030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.702038 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.703395 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.703799 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.703800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.703800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.703865 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.704038 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.704039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.704039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.704138 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.704184 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.704318 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.704387 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:13:11.704520 2623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.698616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:53414)
0807 20:13:11.698730 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.698741 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.698816 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.698947 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.698948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.699149 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.699196 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.699198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.699815 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.699815 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.700692 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.700694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.701014 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.701015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.701081 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.701540 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.701545 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.701927 (+ 382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.703494 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.703495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.703496 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.703722 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.703920 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.703921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.703921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.703988 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.704187 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.704188 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.704233 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.704303 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.704409 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":71,"threads_started":1}
I20260807 20:13:11.704828 1731 tablet_copy_service.cc:140] P 9859649d8d244f6983ca11af92fdb052: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d80a1bea047e480394df4430b235c8aa ({username='oryx', principal='oryx/127.23.200.253@KRBTEST.COM'} at 127.0.0.1:53414)
I20260807 20:13:11.704934 1731 tablet_copy_service.cc:161] P 9859649d8d244f6983ca11af92fdb052: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d80a1bea047e480394df4430b235c8aa at {username='oryx', principal='oryx/127.23.200.253@KRBTEST.COM'} at 127.0.0.1:53414: session id = d80a1bea047e480394df4430b235c8aa-00000000000000000000000000000000
I20260807 20:13:11.705801 1731 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 20:13:11.706509 2448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 20:13:11.707422 2448 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: tablet copy: Starting download of 0 data blocks...
I20260807 20:13:11.707564 2448 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: tablet copy: Starting download of 1 WAL segments...
I20260807 20:13:11.708463 2448 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 20:13:11.709249 1731 tablet_copy_service.cc:342] P 9859649d8d244f6983ca11af92fdb052: Request end of tablet copy session d80a1bea047e480394df4430b235c8aa-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.23.200.253@KRBTEST.COM'} at 127.0.0.1:53414
I20260807 20:13:11.709297 1731 tablet_copy_service.cc:433] P 9859649d8d244f6983ca11af92fdb052: ending tablet copy session d80a1bea047e480394df4430b235c8aa-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d80a1bea047e480394df4430b235c8aa
I20260807 20:13:11.711815 2448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:11.712968 2624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:11.713001 2625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:11.713165 2627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:11.713229 2448 server_base.cc:1034] running on GCE node
I20260807 20:13:11.713361 2448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:11.713524 2448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:11.714648 2448 hybrid_clock.cc:648] HybridClock initialized: now 1786133591714636 us; error 29 us; skew 500 ppm
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133591, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:11.715958 2448 init.cc:377] Logged in from keytab as oryx/127.23.200.253@KRBTEST.COM (short username oryx)
I20260807 20:13:11.716215 2448 webserver.cc:460] Webserver started at http://127.23.200.253:45001/ using document root <none> and password file <none>
I20260807 20:13:11.716333 2448 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:11.716377 2448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:11.716988 2448 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.717628 2637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.717759 2448 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.717809 2448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/wal
uuid: "d80a1bea047e480394df4430b235c8aa"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "ddd7a8ffbd63776b2cfaf0ab3666cfe1"
server_key_iv: "cecc27221a42260d8bece661e1738fc4"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.717867 2448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:11.735540 2448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:11.737249 2448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:11.740105 2448 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.253:42419
I20260807 20:13:11.740126 2735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.253:42419 every 8 connection(s)
I20260807 20:13:11.740422 2448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-1/data/info.pb
I20260807 20:13:11.741427 2736 sys_catalog.cc:263] Verifying existing consensus state
I20260807 20:13:11.741978 2736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: Bootstrap starting.
I20260807 20:13:11.744961 2736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:13:11.745196 2736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: Bootstrap complete.
I20260807 20:13:11.745404 2736 raft_consensus.cc:348] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: NON_VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: true } }
I20260807 20:13:11.745496 2736 raft_consensus.cc:729] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d80a1bea047e480394df4430b235c8aa, State: Initialized, Role: LEARNER
I20260807 20:13:11.745540 2736 consensus_queue.cc:260] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: NON_VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: true } }
I20260807 20:13:11.745733 2736 sys_catalog.cc:564] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:11.745720 2740 sys_catalog.cc:455] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: NON_VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: true } } }
I20260807 20:13:11.745827 2740 sys_catalog.cc:458] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: This master's current role is: LEARNER
I20260807 20:13:11.748344 2767 catalog_manager.cc:1259] Loaded cluster ID: 09fc8a617b444902b2c24ba4c910146c
I20260807 20:13:11.748394 2767 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: loading cluster ID for follower catalog manager: success
I20260807 20:13:11.749441 2767 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: acquiring CA information for follower catalog manager: success
I20260807 20:13:11.749738 2767 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 20:13:11.753916 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.747072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55596 (local address 127.23.200.253:42419)
0807 20:13:11.747220 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.747222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.747230 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:13:11.747251 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.747252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.747261 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.747989 (+ 728us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.748029 (+ 40us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.748990 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.750028 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.750224 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.750277 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.751574 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.751575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.751575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.751581 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.752832 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.753256 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.753257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.753257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.753312 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.753492 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.753492 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.753493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.753581 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.753648 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.753781 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.753847 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for oryx/127.23.200.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.200.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.200.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.252.keytab.
Entry for principal oryx/127.23.200.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.252.keytab.
I20260807 20:13:11.780243 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.252:34687
--webserver_interface=127.23.200.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.252.keytab
--principal=oryx/127.23.200.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:33189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:37725,127.23.200.253:42419,127.23.200.252:34687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:11.863822 2772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:11.864030 2772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:11.864051 2772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:11.865214 2772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:11.865254 2772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:11.865268 2772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:11.865279 2772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:11.865289 2772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:13:11.866818 2772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:33189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:37725,127.23.200.253:42419,127.23.200.252:34687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.252.keytab
--principal=oryx/127.23.200.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.252:34687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.200.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:11.867018 2772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:11.867210 2772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:11.872133 2782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:11.872146 2785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:11.872138 2781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:11.872383 2772 server_base.cc:1034] running on GCE node
I20260807 20:13:11.872695 2772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:11.872999 2772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:11.874135 2772 hybrid_clock.cc:648] HybridClock initialized: now 1786133591874130 us; error 37 us; skew 500 ppm
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133591, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:11.876691 2772 init.cc:377] Logged in from keytab as oryx/127.23.200.252@KRBTEST.COM (short username oryx)
I20260807 20:13:11.877151 2772 webserver.cc:460] Webserver started at http://127.23.200.252:38999/ using document root <none> and password file <none>
I20260807 20:13:11.877355 2772 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:11.877394 2772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:11.877492 2772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:11.878507 2772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/data/instance:
uuid: "cfff710829cd4890b60ca43bc5a008ed"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "9487759e95ba2ae26df98da1ce63b05b"
server_key_iv: "6a449806ea8a7e3191147415569107de"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.878796 2772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/wal/instance:
uuid: "cfff710829cd4890b60ca43bc5a008ed"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "9487759e95ba2ae26df98da1ce63b05b"
server_key_iv: "6a449806ea8a7e3191147415569107de"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.879982 2772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:13:11.880694 2800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.880868 2772 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:13:11.880950 2772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/wal
uuid: "cfff710829cd4890b60ca43bc5a008ed"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "9487759e95ba2ae26df98da1ce63b05b"
server_key_iv: "6a449806ea8a7e3191147415569107de"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.881021 2772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:11.892163 2772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:11.894099 2772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:11.894212 2772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:11.898070 2772 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.252:34687
I20260807 20:13:11.898106 2893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.252:34687 every 8 connection(s)
I20260807 20:13:11.898459 2772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/data/info.pb
I20260807 20:13:11.899427 2894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:11.900916 2894 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:13:11.905848 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 2772
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133591, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.252@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM
I20260807 20:13:11.910092 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.901855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53428 (local address 127.23.200.254:37725)
0807 20:13:11.901909 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.901911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.901995 (+ 84us) server_negotiation.cc:407] Connection header received
0807 20:13:11.902584 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.902585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.902598 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.902655 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.903244 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.903758 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.904738 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.904923 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.904968 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.907380 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.907382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.907382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.907391 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.908712 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.909171 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.909172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.909172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.909236 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.909459 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.909460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.909460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.909595 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.909652 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.909959 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.910027 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:13:11.910307 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.901707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:53428)
0807 20:13:11.901888 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.901926 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.902490 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.902716 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.902720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.902982 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.903173 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.903177 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.903846 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.903847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.904662 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.904665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.904995 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.904996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.905104 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.905526 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.905534 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.907308 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.908782 (+ 1474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.908789 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.908791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.909121 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.909296 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.909297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.909297 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.909390 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.909657 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.909664 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.909806 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.909930 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.910083 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":84,"threads_started":1}
I20260807 20:13:11.911026 2894 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.200.252@KRBTEST.COM
I20260807 20:13:11.914558 2900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.906439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41690 (local address 127.23.200.252:34687)
0807 20:13:11.906585 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.906589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.906605 (+ 16us) server_negotiation.cc:407] Connection header received
0807 20:13:11.906623 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.906625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.906642 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.906725 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.907324 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.908267 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.909052 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.909480 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.909547 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.911857 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.911864 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.911865 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.911871 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.913473 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.913805 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.913808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.913809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.913861 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.914033 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.914034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.914034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.914150 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.914279 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.914362 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.914461 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133591, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.252@KRBTEST.COM for oryx/127.23.200.253@KRBTEST.COM
I20260807 20:13:11.917037 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.911245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:42419 (local address 127.0.0.1:55600)
0807 20:13:11.911293 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.911303 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.911344 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.911496 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.911497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.911627 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.911679 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.911681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.912289 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.912290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.912906 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.912909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.913394 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.913395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.913449 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.913733 (+ 284us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.913738 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.914893 (+ 1155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.916155 (+ 1262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.916156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.916157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.916305 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.916492 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.916493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.916493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.916549 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.916764 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.916765 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.916805 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.916865 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.916951 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 20:13:11.917064 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.911343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55600 (local address 127.23.200.253:42419)
0807 20:13:11.911394 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.911396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.911404 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:13:11.911428 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.911430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.911440 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.911487 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.911745 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.912217 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.912985 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.913323 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.913396 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.914975 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.914976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.914976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.914983 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.916086 (+ 1103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.916374 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.916375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.916376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.916436 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.916599 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.916599 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.916600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.916710 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.916766 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.916895 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.916981 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 20:13:11.917418 2894 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133591, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.252@KRBTEST.COM for oryx/127.23.200.252@KRBTEST.COM
I20260807 20:13:11.922590 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.917633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:34687 (local address 127.0.0.1:41706)
0807 20:13:11.917674 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.917684 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.917722 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.917818 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.917819 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.917922 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.917948 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.917948 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.918494 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.918495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.918983 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.918984 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.919266 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.919266 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.919318 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.919600 (+ 282us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.919603 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.920574 (+ 971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.921914 (+ 1340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.921914 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.921915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.922048 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.922170 (+ 122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.922170 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.922171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.922209 (+ 38us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.922354 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.922355 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.922384 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.922450 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.922515 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 20:13:11.922636 2900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.917686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41706 (local address 127.23.200.252:34687)
0807 20:13:11.917740 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.917741 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.917750 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:11.917766 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.917767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.917776 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.917813 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.917995 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.918440 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.919034 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.919217 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.919264 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.920650 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.920651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.920651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.920657 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.921858 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.922091 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.922092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.922092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.922131 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.922240 (+ 109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.922241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.922241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.922318 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.922356 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.922483 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.922556 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":38}
I20260807 20:13:11.923863 2894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: Bootstrap starting.
I20260807 20:13:11.924688 2894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:11.924944 2894 log.cc:824] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:11.925729 2894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: No bootstrap required, opened a new log
I20260807 20:13:11.927232 2894 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } }
I20260807 20:13:11.927350 2894 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:11.927371 2894 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfff710829cd4890b60ca43bc5a008ed, State: Initialized, Role: FOLLOWER
I20260807 20:13:11.927489 2894 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } }
I20260807 20:13:11.927796 2894 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:11.927762 2905 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } } }
I20260807 20:13:11.927857 2905 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [sys.catalog]: This master's current role is: FOLLOWER
W20260807 20:13:11.929320 2927 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:13:11.929373 2927 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:13:11.931458 2772 master_runner.cc:428] Detected that this master cfff710829cd4890b60ca43bc5a008ed is joining an existing cluster
I20260807 20:13:11.931499 2772 master_runner.cc:432] Initiating AddMaster RPC to add 127.23.200.252:34687
I20260807 20:13:11.939245 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.932454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.23.200.254:37725)
0807 20:13:11.932500 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.932502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.932717 (+ 215us) server_negotiation.cc:407] Connection header received
0807 20:13:11.932815 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.932816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.932828 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.932857 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.933468 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.934225 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.935385 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.935688 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.935723 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.936772 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.936774 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.936774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.936782 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.938183 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.938516 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.938518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.938518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.938577 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.938744 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.938745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.938746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.938860 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.938928 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.939066 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.939149 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:13:11.939266 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.932403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:53436)
0807 20:13:11.932651 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.932662 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.932744 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.932935 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.932936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.933288 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.933364 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.933366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.934314 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.934315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.935299 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.935301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.935806 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.935806 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.935871 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.936323 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.936328 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.936687 (+ 359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.938261 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.938262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.938262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.938453 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.938630 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.938631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.938631 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.938693 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.938914 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.938915 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.938974 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.939034 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.939149 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":69,"thread_start_us":83,"threads_started":1}
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133590, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.254@KRBTEST.COM for oryx/127.23.200.252@KRBTEST.COM
I20260807 20:13:11.956223 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.941257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:34687 (local address 127.0.0.1:41728)
0807 20:13:11.941305 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.941314 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.941347 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.941636 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.941638 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.941786 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.941826 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.941828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.942503 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.942504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.943017 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.943018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.943355 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.943356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.943420 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.943802 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.943806 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.945172 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.955125 (+ 9953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.955126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.955127 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.955357 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.955581 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.955582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.955582 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.955656 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.955955 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.955956 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.956007 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.956053 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.956151 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 20:13:11.956657 2943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.941338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41728 (local address 127.23.200.252:34687)
0807 20:13:11.941506 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.941508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.941513 (+ 5us) server_negotiation.cc:407] Connection header received
0807 20:13:11.941543 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.941544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.941552 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.941615 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.941904 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.942396 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.943086 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.943298 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.943349 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.945270 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.945271 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.945272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.945279 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.955036 (+ 9757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.955421 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.955423 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.955423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.955495 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.955732 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.955733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.955733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.955875 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.956422 (+ 547us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.956464 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.956549 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":118,"threads_started":1}
I20260807 20:13:11.957230 1692 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.23.200.252:34687: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed"
member_type: NON_VOTER
last_known_addr {
host: "127.23.200.252"
port: 34687
}
attrs {
promote: true
}
}
dest_uuid: "9859649d8d244f6983ca11af92fdb052"
cas_config_opid_index: 5
I20260807 20:13:11.957518 1692 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: NON_VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: true } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } }
I20260807 20:13:11.957957 2602 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER cfff710829cd4890b60ca43bc5a008ed (127.23.200.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: NON_VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: true } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } } }
I20260807 20:13:11.958182 2900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.932841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41714 (local address 127.23.200.252:34687)
0807 20:13:11.932871 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.932873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.932998 (+ 125us) server_negotiation.cc:407] Connection header received
0807 20:13:11.933067 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.933068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.933074 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.933139 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.933516 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.934239 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.935381 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.935650 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.935704 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.954536 (+ 18832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.954538 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.954538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.954545 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.956399 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.957237 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.957237 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.957238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.957289 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.957592 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.957593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.957594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.957705 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.957745 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.958035 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.958109 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:13:11.958616 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.932479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55614 (local address 127.23.200.253:42419)
0807 20:13:11.932786 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.932788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.932854 (+ 66us) server_negotiation.cc:407] Connection header received
0807 20:13:11.932953 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.932954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.932965 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.932994 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.933508 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.934239 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.935399 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.935698 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.935740 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.954946 (+ 19206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.954948 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.954949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.954958 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.957136 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.957704 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.957705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.957706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.957765 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.958073 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.958074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.958075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.958205 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.958245 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.958440 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.958521 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283}
I20260807 20:13:11.958936 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.932653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:34687 (local address 127.0.0.1:41714)
0807 20:13:11.932944 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.932953 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.933015 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.933187 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.933187 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.933397 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.933448 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.933449 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.934313 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.934314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.935295 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.935298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.935809 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.935809 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.935866 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.953632 (+ 17766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.953637 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.954424 (+ 787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.956557 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.956558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.956559 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.957169 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.957467 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.957468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.957468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.957530 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.957881 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.957882 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.957937 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.958469 (+ 532us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.958566 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":29,"thread_start_us":85,"threads_started":1}
I20260807 20:13:11.959004 2602 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.23.200.252:34687
I20260807 20:13:11.959072 2602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9859649d8d244f6983ca11af92fdb052" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: NON_VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: true } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } } }
I20260807 20:13:11.959132 2602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:11.959384 2772 master.cc:537] Master@127.23.200.252:34687 shutting down...
I20260807 20:13:11.958945 2941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.932492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:42419 (local address 127.0.0.1:55614)
0807 20:13:11.932789 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.932799 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.932882 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.933053 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.933054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.933329 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.933403 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.933405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.934334 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.934334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.935303 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.935306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.935820 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.935820 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.935889 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.953909 (+ 18020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.953914 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.954812 (+ 898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.957402 (+ 2590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.957402 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.957403 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.957615 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.957937 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.957938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.957938 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.957999 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.958287 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.958288 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.958329 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.958740 (+ 411us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.958843 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":44,"thread_start_us":86,"threads_started":1}
W20260807 20:13:11.959743 1669 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 -> Peer cfff710829cd4890b60ca43bc5a008ed (127.23.200.252:34687): Couldn't send request to peer cfff710829cd4890b60ca43bc5a008ed. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 20:13:11.962747 2772 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:11.962854 2772 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:11.962940 2772 tablet_replica.cc:330] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: stopping tablet replica
I20260807 20:13:11.963245 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.959110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:42419 (local address 127.0.0.1:55628)
0807 20:13:11.959258 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.959268 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.959314 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.960104 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.960106 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.960107 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:11.960150 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.960152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.961044 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.961045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.962177 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.962180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.962985 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.962986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.963059 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.963085 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.963135 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.963176 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260807 20:13:11.963313 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.959327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55628 (local address 127.23.200.253:42419)
0807 20:13:11.959825 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.959827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.959834 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:13:11.959853 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.959854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.959878 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.959936 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:11.960222 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.960953 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.962279 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.962896 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.962969 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.963056 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.963213 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.963255 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474}
I20260807 20:13:11.963522 2683 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 LEARNER]: Refusing update from remote peer 9859649d8d244f6983ca11af92fdb052: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 20:13:11.963727 2604 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: NON_VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 20:13:11.964345 2683 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER cfff710829cd4890b60ca43bc5a008ed (127.23.200.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: NON_VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: true } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } } }
I20260807 20:13:11.965602 2740 sys_catalog.cc:455] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9859649d8d244f6983ca11af92fdb052. Latest consensus state: current_term: 1 leader_uuid: "9859649d8d244f6983ca11af92fdb052" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: NON_VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: true } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } } }
I20260807 20:13:11.965682 2740 sys_catalog.cc:458] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: This master's current role is: LEARNER
I20260807 20:13:11.965711 2740 sys_catalog.cc:455] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9859649d8d244f6983ca11af92fdb052" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: NON_VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: true } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } } }
I20260807 20:13:11.965749 2740 sys_catalog.cc:458] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: This master's current role is: LEARNER
I20260807 20:13:11.965811 2772 master.cc:552] Master@127.23.200.252:34687 shutdown complete.
I20260807 20:13:11.965840 2772 master_runner.cc:305] Clearing existing system tablet
I20260807 20:13:11.965986 2772 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:11.966033 2772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:11.966182 2604 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052: attempting to promote NON_VOTER d80a1bea047e480394df4430b235c8aa to VOTER
I20260807 20:13:11.966380 2604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } }
I20260807 20:13:11.966980 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.966816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:34687 (local address 127.0.0.1:41740)
0807 20:13:11.966910 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.966952 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:34687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 20:13:11.966991 2772 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:13:11.967096 2683 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 LEARNER]: Refusing update from remote peer 9859649d8d244f6983ca11af92fdb052: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 20:13:11.967315 2948 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 20:13:11.967479 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.967374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:34687 (local address 127.0.0.1:41750)
0807 20:13:11.967427 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.967457 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:34687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 20:13:11.967561 2949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.967655 2772 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:11.967717 2772 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.967743 2772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/wal
uuid: "cfff710829cd4890b60ca43bc5a008ed"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "9487759e95ba2ae26df98da1ce63b05b"
server_key_iv: "6a449806ea8a7e3191147415569107de"
server_key_version: "encryptionkey@0"
W20260807 20:13:11.967948 1669 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 -> Peer cfff710829cd4890b60ca43bc5a008ed (127.23.200.252:34687): Couldn't send request to peer cfff710829cd4890b60ca43bc5a008ed. Status: Network error: Client connection negotiation failed: client connection to 127.23.200.252:34687: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 20:13:11.968004 2602 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d80a1bea047e480394df4430b235c8aa (127.23.200.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } } }
I20260807 20:13:11.968524 2772 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:11.968951 2602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9859649d8d244f6983ca11af92fdb052" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } } }
I20260807 20:13:11.969022 2602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:11.969292 2604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9859649d8d244f6983ca11af92fdb052" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } } }
I20260807 20:13:11.969370 2604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:11.969455 2683 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d80a1bea047e480394df4430b235c8aa (127.23.200.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } } }
I20260807 20:13:11.969901 2772 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 20:13:11.969961 2772 log.cc:1196] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 20:13:11.970150 2772 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: Deleting consensus metadata
I20260807 20:13:11.970381 2772 master_runner.cc:315] Copying system tablet from 127.23.200.254:37725
I20260807 20:13:11.970338 2946 sys_catalog.cc:455] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9859649d8d244f6983ca11af92fdb052" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } } }
I20260807 20:13:11.970427 2946 sys_catalog.cc:458] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:13:11.970956 2772 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: tablet copy: Beginning tablet copy session from remote peer at address 127.23.200.254:37725
I20260807 20:13:11.977061 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.971175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:53438)
0807 20:13:11.971327 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:11.971336 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:11.971386 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:11.971506 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:11.971507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:11.971661 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:11.971713 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.971715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.972606 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.972606 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.973197 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:11.973199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.973633 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.973633 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:11.973679 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.974102 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:11.974107 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:11.974392 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:11.976109 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.976110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.976111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.976285 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.976490 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:11.976490 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:11.976491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:11.976563 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:11.976793 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:11.976794 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:11.976849 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 20:13:11.976895 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 20:13:11.976981 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":73,"threads_started":1}
I20260807 20:13:11.977136 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:11.971244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53438 (local address 127.23.200.254:37725)
0807 20:13:11.971290 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:11.971292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:11.971390 (+ 98us) server_negotiation.cc:407] Connection header received
0807 20:13:11.971426 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:11.971427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:11.971437 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:11.971502 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:11.971768 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.972537 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.973260 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:11.973571 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:11.973635 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:11.974479 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:11.974480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:11.974481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:11.974488 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:11.976011 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.976356 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.976358 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.976359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.976428 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:11.976614 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:11.976614 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:11.976615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:11.976741 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:11.976807 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:11.976932 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 20:13:11.977009 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:13:11.977290 1731 tablet_copy_service.cc:140] P 9859649d8d244f6983ca11af92fdb052: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cfff710829cd4890b60ca43bc5a008ed ({username='oryx', principal='oryx/127.23.200.252@KRBTEST.COM'} at 127.0.0.1:53438)
I20260807 20:13:11.977330 1731 tablet_copy_service.cc:161] P 9859649d8d244f6983ca11af92fdb052: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cfff710829cd4890b60ca43bc5a008ed at {username='oryx', principal='oryx/127.23.200.252@KRBTEST.COM'} at 127.0.0.1:53438: session id = cfff710829cd4890b60ca43bc5a008ed-00000000000000000000000000000000
I20260807 20:13:11.977900 1731 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 20:13:11.978446 2772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 20:13:11.979745 2772 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: tablet copy: Starting download of 0 data blocks...
I20260807 20:13:11.979892 2772 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: tablet copy: Starting download of 1 WAL segments...
I20260807 20:13:11.980716 2772 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 20:13:11.981642 1731 tablet_copy_service.cc:342] P 9859649d8d244f6983ca11af92fdb052: Request end of tablet copy session cfff710829cd4890b60ca43bc5a008ed-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.23.200.252@KRBTEST.COM'} at 127.0.0.1:53438
I20260807 20:13:11.981698 1731 tablet_copy_service.cc:433] P 9859649d8d244f6983ca11af92fdb052: ending tablet copy session cfff710829cd4890b60ca43bc5a008ed-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cfff710829cd4890b60ca43bc5a008ed
I20260807 20:13:11.984159 2772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:11.985363 2959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:11.985436 2960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:11.985561 2772 server_base.cc:1034] running on GCE node
W20260807 20:13:11.985457 2965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:11.985707 2772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:11.985846 2772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:11.986966 2772 hybrid_clock.cc:648] HybridClock initialized: now 1786133591986956 us; error 30 us; skew 500 ppm
Aug 07 20:13:11 dist-test-slave-2j7r krb5kdc[1624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133591, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:11.988343 2772 init.cc:377] Logged in from keytab as oryx/127.23.200.252@KRBTEST.COM (short username oryx)
I20260807 20:13:11.988607 2772 webserver.cc:460] Webserver started at http://127.23.200.252:44943/ using document root <none> and password file <none>
I20260807 20:13:11.988713 2772 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:11.988747 2772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:11.989488 2772 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:11.989981 2977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.990097 2772 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:11.990140 2772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/wal
uuid: "cfff710829cd4890b60ca43bc5a008ed"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "9487759e95ba2ae26df98da1ce63b05b"
server_key_iv: "6a449806ea8a7e3191147415569107de"
server_key_version: "encryptionkey@0"
I20260807 20:13:11.990187 2772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:11.995544 2772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:11.996794 2772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:11.999797 2772 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.252:34687
I20260807 20:13:12.000223 2772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/master-2/data/info.pb
I20260807 20:13:12.000241 3069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.252:34687 every 8 connection(s)
I20260807 20:13:12.001055 3070 sys_catalog.cc:263] Verifying existing consensus state
I20260807 20:13:12.001573 3070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: Bootstrap starting.
I20260807 20:13:12.004683 3070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:13:12.004879 3070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: Bootstrap complete.
I20260807 20:13:12.005079 3070 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } }
I20260807 20:13:12.005231 3070 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cfff710829cd4890b60ca43bc5a008ed, State: Initialized, Role: LEARNER
I20260807 20:13:12.005275 3070 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } }
I20260807 20:13:12.005466 3070 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:12.005461 3075 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } } }
I20260807 20:13:12.005556 3075 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [sys.catalog]: This master's current role is: LEARNER
I20260807 20:13:12.007015 3102 catalog_manager.cc:1259] Loaded cluster ID: 09fc8a617b444902b2c24ba4c910146c
I20260807 20:13:12.007537 3102 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: loading cluster ID for follower catalog manager: success
I20260807 20:13:12.008351 3102 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: acquiring CA information for follower catalog manager: success
I20260807 20:13:12.008613 3102 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 20:13:12.023939 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.017230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41764 (local address 127.23.200.252:34687)
0807 20:13:12.017350 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.017352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.017358 (+ 6us) server_negotiation.cc:407] Connection header received
0807 20:13:12.017379 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.017379 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.017388 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.017481 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:12.018041 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.018754 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.019431 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.019677 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.019729 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.020920 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:12.020922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:12.020922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:12.020930 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:12.022889 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.023213 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.023214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.023215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.023275 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.023437 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.023437 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.023437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.023547 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:12.023635 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.023732 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.023811 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
I20260807 20:13:12.024472 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 1801
I20260807 20:13:12.037447 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:44069
--local_ip_for_outbound_sockets=127.23.200.193
--tserver_master_addrs=127.23.200.254:37725
--webserver_port=36801
--webserver_interface=127.23.200.193
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.193.keytab
--principal=oryx/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:33189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.200.254:37725,127.23.200.253:42419
--tserver_master_addrs=127.23.200.254:37725,127.23.200.253:42419,127.23.200.252:34687 with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:12.120358 3104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:12.120630 3104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:12.120687 3104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:12.121932 3104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:12.121976 3104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:12.122001 3104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:13:12.123534 3104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:33189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.193.keytab
--principal=oryx/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:44069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=36801
--tserver_master_addrs=127.23.200.254:37725,127.23.200.253:42419,127.23.200.252:34687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:12.123767 3104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:12.124004 3104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:12.129043 3109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:12.129043 3117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:12.129040 3113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:12.129495 3104 server_base.cc:1034] running on GCE node
I20260807 20:13:12.129720 3104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:12.129948 3104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:12.131120 3104 hybrid_clock.cc:648] HybridClock initialized: now 1786133592131101 us; error 27 us; skew 500 ppm
Aug 07 20:13:12 dist-test-slave-2j7r krb5kdc[1624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133592, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:12.133744 3104 init.cc:377] Logged in from keytab as oryx/127.23.200.193@KRBTEST.COM (short username oryx)
I20260807 20:13:12.134181 3104 webserver.cc:460] Webserver started at http://127.23.200.193:36801/ using document root <none> and password file <none>
I20260807 20:13:12.134379 3104 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:12.134452 3104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:12.135878 3104 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:12.136485 3131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:12.136647 3104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:12.136763 3104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "c0b79210210043cd8434b1c3e065f28c"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "a6cfadad3ec028f0b4711ca80e561272"
server_key_iv: "d31bb957e77909de33f90901cda74ecc"
server_key_version: "encryptionkey@0"
I20260807 20:13:12.137141 3104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:12.158784 3104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:12.160892 3104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:12.161029 3104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:12.161270 3104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:12.161585 3104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:12.161628 3104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:12.161660 3104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:12.161684 3104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:12.167618 3104 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:44069
I20260807 20:13:12.167665 3334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:44069 every 8 connection(s)
I20260807 20:13:12.168021 3104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:13:12.172864 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 3104
I20260807 20:13:12.173063 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 2018
Aug 07 20:13:12 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133592, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.252@KRBTEST.COM
I20260807 20:13:12.178830 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.168720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:45417 (local address 127.23.200.252:34687)
0807 20:13:12.168865 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.168867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.169126 (+ 259us) server_negotiation.cc:407] Connection header received
0807 20:13:12.170315 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.170316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.170332 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.170388 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:12.171035 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.171542 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.172839 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.173181 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.173236 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.176289 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:12.176290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:12.176290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:12.176297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:12.177575 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.177979 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.177980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.177980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.178032 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.178236 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.178237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.178237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.178342 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:12.178374 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.178666 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.178751 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260807 20:13:12.179291 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.168733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:34687 (local address 127.23.200.193:45417)
0807 20:13:12.168995 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:12.169035 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:12.170016 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:12.170391 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:12.170392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:12.170687 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:12.170923 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.170930 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.171641 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.171642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.172733 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.172736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.173296 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.173297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.173427 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.174163 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:12.174176 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:12.176196 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:12.177655 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.177662 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.177665 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.177899 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.178085 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.178086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.178086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.178186 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.178406 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:12.178410 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:12.178551 (+ 141us) client_negotiation.cc:769] Sending connection context
0807 20:13:12.178947 (+ 396us) client_negotiation.cc:240] Negotiation successful
0807 20:13:12.179095 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":6,"thread_start_us":212,"threads_started":1}
I20260807 20:13:12.194934 3335 heartbeater.cc:347] Connected to a master server at 127.23.200.252:34687
I20260807 20:13:12.195118 3335 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:12.195405 3335 heartbeater.cc:511] Master 127.23.200.252:34687 requested a full tablet report, sending...
I20260807 20:13:12.195946 3014 ts_manager.cc:194] Registered new tserver with Master: c0b79210210043cd8434b1c3e065f28c (127.23.200.193:44069)
Aug 07 20:13:12 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133592, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM
Aug 07 20:13:12 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133592, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.253@KRBTEST.COM
I20260807 20:13:12.200634 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:46497
--local_ip_for_outbound_sockets=127.23.200.194
--tserver_master_addrs=127.23.200.254:37725
--webserver_port=44183
--webserver_interface=127.23.200.194
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.194.keytab
--principal=oryx/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:33189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.200.254:37725,127.23.200.253:42419
--tserver_master_addrs=127.23.200.254:37725,127.23.200.253:42419,127.23.200.252:34687 with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:13:12.201413 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.168913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:39225 (local address 127.23.200.253:42419)
0807 20:13:12.169047 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.169049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.169179 (+ 130us) server_negotiation.cc:407] Connection header received
0807 20:13:12.170183 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.170184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.170198 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.170251 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:12.171040 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.171780 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.172824 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.173160 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.173222 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.198781 (+ 25559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:12.198783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:12.198783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:12.198791 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:12.200101 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.200596 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.200597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.200597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.200652 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.200820 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.200821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.200822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.200975 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:12.201038 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.201239 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.201314 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260807 20:13:12.201442 3342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.168848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:42419 (local address 127.23.200.193:39225)
0807 20:13:12.169123 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:12.169131 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:12.170025 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:12.170294 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:12.170299 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:12.170694 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:12.170923 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.170930 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.171857 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.171858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.172733 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.172736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.196466 (+ 23730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.196467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.196547 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.196918 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:12.196923 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:12.198665 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:12.200185 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.200186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.200187 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.200494 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.200711 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.200712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.200712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.200770 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.201066 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:12.201067 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:12.201135 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 20:13:12.201198 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 20:13:12.201316 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":58,"thread_start_us":143,"threads_started":1}
I20260807 20:13:12.201484 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.168721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:53247 (local address 127.23.200.254:37725)
0807 20:13:12.168769 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.168772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.169153 (+ 381us) server_negotiation.cc:407] Connection header received
0807 20:13:12.170130 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.170131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.170146 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.170199 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:12.171015 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.171782 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.195876 (+ 24094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.196125 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.196174 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.198283 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:12.198285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:12.198286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:12.198295 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:12.200029 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.200594 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.200595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.200596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.200670 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.200878 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.200880 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.200880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.201035 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:12.201115 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.201313 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.201401 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:13:12.201983 3337 heartbeater.cc:347] Connected to a master server at 127.23.200.253:42419
I20260807 20:13:12.202067 3337 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:12.202150 3337 heartbeater.cc:511] Master 127.23.200.253:42419 requested a full tablet report, sending...
I20260807 20:13:12.202328 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.168733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.193:53247)
0807 20:13:12.169064 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:12.169074 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:12.170028 (+ 954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:12.170295 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:12.170299 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:12.170687 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:12.170922 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.170928 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.171889 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.171890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.172616 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.172618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.196265 (+ 23647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.196267 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.196339 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.196786 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:12.196791 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:12.198160 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:12.200223 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.200224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.200225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.200501 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.200751 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.200752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.200753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.200814 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.201135 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:12.201136 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:12.201186 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 20:13:12.201791 (+ 605us) client_negotiation.cc:240] Negotiation successful
0807 20:13:12.201901 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":120,"thread_start_us":115,"threads_started":1}
I20260807 20:13:12.202602 2670 ts_manager.cc:194] Registered new tserver with Master: c0b79210210043cd8434b1c3e065f28c (127.23.200.193:44069)
I20260807 20:13:12.202740 3336 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37725
I20260807 20:13:12.202785 3336 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:12.202873 3336 heartbeater.cc:511] Master 127.23.200.254:37725 requested a full tablet report, sending...
I20260807 20:13:12.203135 1692 ts_manager.cc:194] Re-registered known tserver with Master: c0b79210210043cd8434b1c3e065f28c (127.23.200.193:44069)
I20260807 20:13:12.203514 1692 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:53247
W20260807 20:13:12.284339 3345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:12.284551 3345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:12.284574 3345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:12.285758 3345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:12.285799 3345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:12.285828 3345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:13:12.287320 3345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:33189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.194.keytab
--principal=oryx/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:46497
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=44183
--tserver_master_addrs=127.23.200.254:37725,127.23.200.253:42419,127.23.200.252:34687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:12.287563 3345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:12.287788 3345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:12.292742 3357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:12.292742 3358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:12.292773 3360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:12.292851 3345 server_base.cc:1034] running on GCE node
I20260807 20:13:12.293161 3345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:12.293380 3345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:12.294545 3345 hybrid_clock.cc:648] HybridClock initialized: now 1786133592294530 us; error 30 us; skew 500 ppm
Aug 07 20:13:12 dist-test-slave-2j7r krb5kdc[1624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133592, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:12.296934 3345 init.cc:377] Logged in from keytab as oryx/127.23.200.194@KRBTEST.COM (short username oryx)
I20260807 20:13:12.297381 3345 webserver.cc:460] Webserver started at http://127.23.200.194:44183/ using document root <none> and password file <none>
I20260807 20:13:12.297619 3345 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:12.297684 3345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:12.298991 3345 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:13:12.299778 3370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:12.299983 3345 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:13:12.300081 3345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "8ba1c663c66d4941ba2bc5fb0f482053"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "5c719eb02d7ee29d07cfebbc5452f175"
server_key_iv: "e80c660b3046674eba274f7eb2f2ddae"
server_key_version: "encryptionkey@0"
I20260807 20:13:12.300444 3345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:12.322902 3345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:12.324987 3345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:12.325127 3345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:12.325333 3345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:12.325654 3345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:12.325690 3345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:12.325711 3345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:12.325733 3345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:12.331384 3345 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:46497
I20260807 20:13:12.331449 3567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:46497 every 8 connection(s)
I20260807 20:13:12.331785 3345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:13:12.336871 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 3345
I20260807 20:13:12.337046 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 2231
Aug 07 20:13:12 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133592, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.252@KRBTEST.COM
I20260807 20:13:12.341928 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.332532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:59489 (local address 127.23.200.252:34687)
0807 20:13:12.332590 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.332592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.333134 (+ 542us) server_negotiation.cc:407] Connection header received
0807 20:13:12.333971 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.333973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.333987 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.334017 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:12.334771 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.335520 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.336668 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.337000 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.337063 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.339440 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:12.339441 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:12.339442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:12.339449 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:12.340671 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.341041 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.341042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.341043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.341112 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.341315 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.341315 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.341316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.341433 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:12.341468 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.341763 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.341837 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:13:12.342409 3583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.332544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:34687 (local address 127.23.200.194:59489)
0807 20:13:12.333034 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:12.333044 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:12.333857 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:12.334099 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:12.334100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:12.334459 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:12.334658 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.334665 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.335618 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.335619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.336577 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.336580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.337251 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.337252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.337377 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.337849 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:12.337859 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:12.339346 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:12.340745 (+ 1399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.340751 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.340753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.340977 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.341169 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.341169 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.341170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.341266 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.341504 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:12.341509 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:12.341649 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 20:13:12.342045 (+ 396us) client_negotiation.cc:240] Negotiation successful
0807 20:13:12.342211 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":4,"thread_start_us":107,"threads_started":1}
I20260807 20:13:12.351207 3568 heartbeater.cc:347] Connected to a master server at 127.23.200.252:34687
I20260807 20:13:12.351334 3568 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:12.351557 3568 heartbeater.cc:511] Master 127.23.200.252:34687 requested a full tablet report, sending...
I20260807 20:13:12.352245 3014 ts_manager.cc:194] Registered new tserver with Master: 8ba1c663c66d4941ba2bc5fb0f482053 (127.23.200.194:46497)
Aug 07 20:13:12 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133592, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.253@KRBTEST.COM
Aug 07 20:13:12 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133592, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM
I20260807 20:13:12.355609 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.332531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:54801 (local address 127.23.200.253:42419)
0807 20:13:12.332590 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.332593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.332882 (+ 289us) server_negotiation.cc:407] Connection header received
0807 20:13:12.333960 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.333962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.333975 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.334023 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:12.334735 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.335523 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.350537 (+ 15014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.350842 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.350923 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.353049 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:12.353052 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:12.353052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:12.353060 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:12.354540 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.354824 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.354825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.354825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.354881 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.355065 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.355066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.355066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.355204 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:12.355242 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.355440 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.355522 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 20:13:12.355616 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:43335
--local_ip_for_outbound_sockets=127.23.200.195
--tserver_master_addrs=127.23.200.254:37725
--webserver_port=34073
--webserver_interface=127.23.200.195
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.195.keytab
--principal=oryx/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:33189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.200.254:37725,127.23.200.253:42419
--tserver_master_addrs=127.23.200.254:37725,127.23.200.253:42419,127.23.200.252:34687 with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:13:12.355918 3580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.332543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:42419 (local address 127.23.200.194:54801)
0807 20:13:12.332745 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:12.332789 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:12.333849 (+ 1060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:12.334096 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:12.334100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:12.334433 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:12.334657 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.334661 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.336676 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.336677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.350392 (+ 13715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.350398 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.350928 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.350929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.351004 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.351471 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:12.351477 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:12.352942 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:12.354610 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.354611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.354611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.354761 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.354933 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.354933 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.354934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.354988 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.355281 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:12.355281 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:12.355315 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 20:13:12.355727 (+ 412us) client_negotiation.cc:240] Negotiation successful
0807 20:13:12.355830 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":104,"threads_started":1}
I20260807 20:13:12.356236 3573 heartbeater.cc:347] Connected to a master server at 127.23.200.253:42419
I20260807 20:13:12.356297 3573 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:12.356371 3573 heartbeater.cc:511] Master 127.23.200.253:42419 requested a full tablet report, sending...
I20260807 20:13:12.356407 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.333121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:50485 (local address 127.23.200.254:37725)
0807 20:13:12.333170 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.333171 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.333182 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:13:12.333973 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.333974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.333987 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.334017 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:12.335616 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.336096 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.336872 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.337166 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.337206 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.353858 (+ 16652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:12.353860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:12.353860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:12.353868 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:12.355294 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.355609 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.355611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.355611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.355671 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.355875 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.355875 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.355876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.355997 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:12.356035 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.356235 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.356316 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 20:13:12.356709 3582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.332884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.194:50485)
0807 20:13:12.333117 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:12.333125 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:12.333857 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:12.334099 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:12.334100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:12.334463 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:12.334658 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.334665 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.336180 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.336181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.336776 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.336778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.351859 (+ 15081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.351860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.351928 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.352294 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:12.352299 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:12.353781 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:12.355387 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.355388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.355388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.355544 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.355746 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.355747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.355747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.355809 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.356085 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:12.356086 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:12.356130 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 20:13:12.356505 (+ 375us) client_negotiation.cc:240] Negotiation successful
0807 20:13:12.356601 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260807 20:13:12.356746 2670 ts_manager.cc:194] Registered new tserver with Master: 8ba1c663c66d4941ba2bc5fb0f482053 (127.23.200.194:46497)
I20260807 20:13:12.357311 3570 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37725
I20260807 20:13:12.357365 3570 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:12.357426 3570 heartbeater.cc:511] Master 127.23.200.254:37725 requested a full tablet report, sending...
I20260807 20:13:12.357694 1692 ts_manager.cc:194] Re-registered known tserver with Master: 8ba1c663c66d4941ba2bc5fb0f482053 (127.23.200.194:46497)
I20260807 20:13:12.358052 1692 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:50485
W20260807 20:13:12.439538 3585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:12.439775 3585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:12.439798 3585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:12.440995 3585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:12.441037 3585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:12.441062 3585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:13:12.442586 3585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:33189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.195.keytab
--principal=oryx/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:43335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=34073
--tserver_master_addrs=127.23.200.254:37725,127.23.200.253:42419,127.23.200.252:34687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:12.442826 3585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:12.443059 3585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:12.448033 3591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:12.448081 3592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:12.448109 3595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:12.448238 3585 server_base.cc:1034] running on GCE node
I20260807 20:13:12.448442 3585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:12.448664 3585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:12.449797 3585 hybrid_clock.cc:648] HybridClock initialized: now 1786133592449785 us; error 29 us; skew 500 ppm
Aug 07 20:13:12 dist-test-slave-2j7r krb5kdc[1624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133592, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:12.452265 3585 init.cc:377] Logged in from keytab as oryx/127.23.200.195@KRBTEST.COM (short username oryx)
I20260807 20:13:12.452663 3585 webserver.cc:460] Webserver started at http://127.23.200.195:34073/ using document root <none> and password file <none>
I20260807 20:13:12.452849 3585 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:12.452903 3585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:12.454295 3585 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:12.454825 3607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:12.454977 3585 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:12.455065 3585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "6f7ecf2c69394d2eb92172f67b98d8aa"
format_stamp: "Formatted at 2026-08-07 20:13:11 on dist-test-slave-2j7r"
server_key: "8c44bdc44cd74fc78b3cb636ea6f7b48"
server_key_iv: "e69120de494d1ebb2719155fe277eb23"
server_key_version: "encryptionkey@0"
I20260807 20:13:12.455379 3585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:12.467118 3585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:12.469385 3585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:12.469504 3585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:12.469732 3585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:12.470022 3585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:12.470059 3585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:12.470080 3585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:12.470096 3585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:12.476083 3585 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:43335
I20260807 20:13:12.476147 3816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:43335 every 8 connection(s)
I20260807 20:13:12.476469 3585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:13:12.481307 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 3585
Aug 07 20:13:12 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133592, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.252@KRBTEST.COM
Aug 07 20:13:12 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133592, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.253@KRBTEST.COM
Aug 07 20:13:12 dist-test-slave-2j7r krb5kdc[1624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133592, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM
I20260807 20:13:12.488394 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.477545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:42419 (local address 127.23.200.195:59939)
0807 20:13:12.477680 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:12.477689 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:12.478019 (+ 330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:12.478324 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:12.478325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:12.478704 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:12.478941 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.478947 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.480449 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.480450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.481120 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.481123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.481671 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.481672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.481813 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.482836 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:12.482840 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:12.485361 (+ 2521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:12.487219 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.487220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.487221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.487424 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.487625 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.487626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.487627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.487685 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.487979 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:12.487980 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:12.488021 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 20:13:12.488081 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 20:13:12.488179 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":4,"thread_start_us":85,"threads_started":1}
I20260807 20:13:12.488410 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.477205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:53639 (local address 127.23.200.252:34687)
0807 20:13:12.477266 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.477268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.477529 (+ 261us) server_negotiation.cc:407] Connection header received
0807 20:13:12.478111 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.478113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.478127 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.478176 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:12.479042 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.479817 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.481362 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.481669 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.481708 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.484656 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:12.484658 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:12.484658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:12.484665 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:12.486458 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.487032 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.487034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.487034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.487093 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.487363 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.487364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.487364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.487495 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:12.487557 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.488245 (+ 688us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.488321 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:13:12.489125 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.477751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:59939 (local address 127.23.200.253:42419)
0807 20:13:12.477800 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.477802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.477811 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:12.478231 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.478233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.478244 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.478298 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:12.479922 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.480375 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.481225 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.481495 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.481540 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.485456 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:12.485457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:12.485457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:12.485464 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:12.487130 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.487503 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.487504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.487504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.487562 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.487750 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.487751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.487751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.487891 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:12.487952 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.488937 (+ 985us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.489013 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:13:12.489161 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.477209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:44883 (local address 127.23.200.254:37725)
0807 20:13:12.477324 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.477326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.477551 (+ 225us) server_negotiation.cc:407] Connection header received
0807 20:13:12.478125 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.478127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.478142 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.478170 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:12.479059 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.479824 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.481362 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.481669 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.481708 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.485353 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:12.485355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:12.485355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:12.485362 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:12.487480 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.488009 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.488010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.488011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.488086 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.488387 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.488388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.488389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.488519 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:12.488593 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.488979 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.489046 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 20:13:12.489900 3823 heartbeater.cc:347] Connected to a master server at 127.23.200.253:42419
I20260807 20:13:12.488550 3826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.477211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:34687 (local address 127.23.200.195:53639)
0807 20:13:12.477424 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:12.477435 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:12.478013 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:12.478571 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:12.478572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:12.478728 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:12.478941 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.478945 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.479949 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.479950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.481262 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.481264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.481858 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.481859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.481916 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.482495 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:12.482505 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:12.484551 (+ 2046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:12.486544 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.486551 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.486553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.486856 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.487160 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.487160 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.487161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.487296 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.487605 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:12.487612 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:12.487790 (+ 178us) client_negotiation.cc:769] Sending connection context
0807 20:13:12.487895 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 20:13:12.488074 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":129,"thread_start_us":88,"threads_started":1}
I20260807 20:13:12.490218 3823 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:12.488896 3829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.477211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.195:44883)
0807 20:13:12.477389 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:12.477431 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:12.478010 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:12.478267 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:12.478270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:12.478698 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:12.478942 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.478947 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.479949 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.479950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.481259 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.481261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.481786 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.481786 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.481847 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.482654 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:12.482659 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:12.485254 (+ 2595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:12.487754 (+ 2500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.487755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.487755 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.487937 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.488241 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:12.488242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:12.488242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:12.488302 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:12.488619 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:12.488620 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:12.488659 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 20:13:12.488709 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 20:13:12.488797 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":67,"thread_start_us":97,"threads_started":1}
I20260807 20:13:12.490446 3823 heartbeater.cc:511] Master 127.23.200.253:42419 requested a full tablet report, sending...
I20260807 20:13:12.490876 2670 ts_manager.cc:194] Registered new tserver with Master: 6f7ecf2c69394d2eb92172f67b98d8aa (127.23.200.195:43335)
I20260807 20:13:12.491647 3822 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37725
I20260807 20:13:12.491700 3822 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:12.491773 3822 heartbeater.cc:511] Master 127.23.200.254:37725 requested a full tablet report, sending...
I20260807 20:13:12.492004 3819 heartbeater.cc:347] Connected to a master server at 127.23.200.252:34687
I20260807 20:13:12.492059 3819 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:12.492059 1692 ts_manager.cc:194] Re-registered known tserver with Master: 6f7ecf2c69394d2eb92172f67b98d8aa (127.23.200.195:43335)
I20260807 20:13:12.492121 3819 heartbeater.cc:511] Master 127.23.200.252:34687 requested a full tablet report, sending...
I20260807 20:13:12.492520 1692 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:44883
I20260807 20:13:12.492828 3014 ts_manager.cc:194] Registered new tserver with Master: 6f7ecf2c69394d2eb92172f67b98d8aa (127.23.200.195:43335)
I20260807 20:13:12.494030 3850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.483506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41766 (local address 127.23.200.252:34687)
0807 20:13:12.483835 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.483837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.483843 (+ 6us) server_negotiation.cc:407] Connection header received
0807 20:13:12.483862 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.483863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.483874 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.484022 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:12.485152 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.485872 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.486927 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.487202 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.487269 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.488955 (+ 1686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:12.488957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:12.488957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:12.488963 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:12.490729 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.491161 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.491162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.491163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.491222 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.493277 (+ 2055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.493278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.493279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.493421 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:12.493519 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.493857 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.493938 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":55,"threads_started":1}
I20260807 20:13:12.495152 3844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.483071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53440 (local address 127.23.200.254:37725)
0807 20:13:12.483193 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.483194 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.483630 (+ 436us) server_negotiation.cc:407] Connection header received
0807 20:13:12.483684 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.483685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.483698 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.484915 (+ 1217us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:12.484954 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.489249 (+ 4295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.490475 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.490798 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.491077 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.492286 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:12.492288 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:12.492288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:12.492294 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:12.494076 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.494523 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.494524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.494524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.494582 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.494724 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.494725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.494725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.494852 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:12.494947 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.495033 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.495093 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
I20260807 20:13:12.496132 3843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.482550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55642 (local address 127.23.200.253:42419)
0807 20:13:12.482671 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.482673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.483445 (+ 772us) server_negotiation.cc:407] Connection header received
0807 20:13:12.483540 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.483541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.483555 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.483604 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:12.484357 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.485136 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.491690 (+ 6554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.491972 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.492031 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.493315 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:12.493316 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:12.493316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:12.493333 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:12.494859 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.495242 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.495243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.495244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.495305 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.495659 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.495660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.495661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.495787 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:12.495884 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.495977 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.496064 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260807 20:13:12.496682 1692 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:13:12.498363 1692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:13:12.506253 3866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.502122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:43335 (local address 127.0.0.1:59866)
0807 20:13:12.502473 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:12.502481 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:12.502514 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:12.502733 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:12.502734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:12.502736 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:12.502803 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.502805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.503868 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.503869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.504739 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.504741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.506020 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.506021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.506098 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.506103 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:13:12.506130 (+ 27us) client_negotiation.cc:240] Negotiation successful
0807 20:13:12.506169 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":89,"threads_started":1}
I20260807 20:13:12.506259 3870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.502046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:44069 (local address 127.0.0.1:55744)
0807 20:13:12.502369 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:12.502379 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:12.502421 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:12.502924 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:12.502925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:12.502927 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:12.502963 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.502964 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.504189 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.504190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.505222 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.505225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.506031 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.506032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.506092 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.506097 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:13:12.506137 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 20:13:12.506180 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":93,"threads_started":1}
I20260807 20:13:12.506302 3875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.502457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55744 (local address 127.23.200.193:44069)
0807 20:13:12.502745 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.502749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.502758 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:12.502805 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.502807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.502824 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.502924 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:12.503019 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.504093 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.505295 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.505942 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.506097 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.506167 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.506193 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.506229 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":80,"threads_started":1}
I20260807 20:13:12.506642 3873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.502214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59866 (local address 127.23.200.195:43335)
0807 20:13:12.502572 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.502577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.502588 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:13:12.502626 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.502628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.502644 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.502723 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:12.502879 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.503775 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.505145 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.505912 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.506402 (+ 490us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.506499 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.506527 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.506566 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":83,"threads_started":1}
I20260807 20:13:12.507210 3867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.502127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:46497 (local address 127.0.0.1:53950)
0807 20:13:12.502574 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:12.502581 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:12.502621 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:12.503081 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:12.503082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:12.503085 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:12.503141 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.503142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.504535 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.504536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.505621 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.505625 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.506615 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.506616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.506686 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.506690 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:13:12.507103 (+ 413us) client_negotiation.cc:240] Negotiation successful
0807 20:13:12.507148 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":92,"threads_started":1}
I20260807 20:13:12.507359 3699 tablet_service.cc:1467] Processing CreateTablet for tablet 5530fa790e3340feb27dc8d90234bb2f (DEFAULT_TABLE table=test-table [id=c20c38286ac3476dad37913de2f07b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:12.507607 3874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.502357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53950 (local address 127.23.200.194:46497)
0807 20:13:12.502548 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.502554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.502615 (+ 61us) server_negotiation.cc:407] Connection header received
0807 20:13:12.502682 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.502684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.502700 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.503391 (+ 691us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:12.503452 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.504458 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.505712 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.506529 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.507399 (+ 870us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.507460 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.507495 (+ 35us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.507535 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":97,"threads_started":1}
I20260807 20:13:12.507507 3213 tablet_service.cc:1467] Processing CreateTablet for tablet 5530fa790e3340feb27dc8d90234bb2f (DEFAULT_TABLE table=test-table [id=c20c38286ac3476dad37913de2f07b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:12.507766 3699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5530fa790e3340feb27dc8d90234bb2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:12.507885 3213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5530fa790e3340feb27dc8d90234bb2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:12.508148 3456 tablet_service.cc:1467] Processing CreateTablet for tablet 5530fa790e3340feb27dc8d90234bb2f (DEFAULT_TABLE table=test-table [id=c20c38286ac3476dad37913de2f07b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:12.508451 3456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5530fa790e3340feb27dc8d90234bb2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:12.510581 3870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.506747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:34687 (local address 127.0.0.1:41780)
0807 20:13:12.506784 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:12.506794 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:12.506840 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:12.507300 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:12.507301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:12.507303 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:12.507332 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.507333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.508178 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.508179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.509210 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.509213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.510372 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.510373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.510436 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.510439 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:13:12.510484 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 20:13:12.510519 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260807 20:13:12.510845 3850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.506837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41780 (local address 127.23.200.252:34687)
0807 20:13:12.506875 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.506877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.506884 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:13:12.506910 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.506911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.506949 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.507009 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:12.507408 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.508086 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.509308 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.509929 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.509998 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.510066 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.510717 (+ 651us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.510767 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:13:12.511159 3030 raft_consensus.cc:1203] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260807 20:13:12.511876 3881 tablet_bootstrap.cc:492] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053: Bootstrap starting.
I20260807 20:13:12.512089 3876 tablet_bootstrap.cc:492] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa: Bootstrap starting.
I20260807 20:13:12.512105 3877 tablet_bootstrap.cc:492] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c: Bootstrap starting.
I20260807 20:13:12.512564 3881 tablet_bootstrap.cc:654] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:12.512635 3877 tablet_bootstrap.cc:654] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:12.512692 3876 tablet_bootstrap.cc:654] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:12.512914 3877 log.cc:824] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:12.512917 3881 log.cc:824] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:12.512997 3876 log.cc:824] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:12.513190 3879 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9859649d8d244f6983ca11af92fdb052. Latest consensus state: current_term: 1 leader_uuid: "9859649d8d244f6983ca11af92fdb052" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } } }
I20260807 20:13:12.513294 3879 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [sys.catalog]: This master's current role is: LEARNER
I20260807 20:13:12.513712 3876 tablet_bootstrap.cc:492] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa: No bootstrap required, opened a new log
I20260807 20:13:12.513779 3876 ts_tablet_manager.cc:1397] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:13:12.514077 3877 tablet_bootstrap.cc:492] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c: No bootstrap required, opened a new log
I20260807 20:13:12.514135 3877 ts_tablet_manager.cc:1397] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 20:13:12.515450 3877 raft_consensus.cc:348] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:12.515574 3877 raft_consensus.cc:374] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:12.515605 3877 raft_consensus.cc:729] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0b79210210043cd8434b1c3e065f28c, State: Initialized, Role: FOLLOWER
I20260807 20:13:12.515583 3876 raft_consensus.cc:348] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:12.515727 3876 raft_consensus.cc:374] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:12.515722 3877 consensus_queue.cc:260] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:12.515767 3876 raft_consensus.cc:729] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7ecf2c69394d2eb92172f67b98d8aa, State: Initialized, Role: FOLLOWER
I20260807 20:13:12.515879 3876 consensus_queue.cc:260] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:12.515980 3877 ts_tablet_manager.cc:1428] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:13:12.516124 3876 ts_tablet_manager.cc:1428] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 20:13:12.516129 3336 heartbeater.cc:503] Master 127.23.200.254:37725 was elected leader, sending a full tablet report...
I20260807 20:13:12.516439 3822 heartbeater.cc:503] Master 127.23.200.254:37725 was elected leader, sending a full tablet report...
I20260807 20:13:12.517966 3881 tablet_bootstrap.cc:492] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053: No bootstrap required, opened a new log
I20260807 20:13:12.518049 3881 ts_tablet_manager.cc:1397] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260807 20:13:12.519840 3881 raft_consensus.cc:348] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:12.519963 3881 raft_consensus.cc:374] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:12.519982 3881 raft_consensus.cc:729] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ba1c663c66d4941ba2bc5fb0f482053, State: Initialized, Role: FOLLOWER
I20260807 20:13:12.520066 3881 consensus_queue.cc:260] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:12.520282 3881 ts_tablet_manager.cc:1428] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:13:12.520344 3570 heartbeater.cc:503] Master 127.23.200.254:37725 was elected leader, sending a full tablet report...
I20260807 20:13:12.546028 3894 raft_consensus.cc:482] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:13:12.546166 3894 raft_consensus.cc:504] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:12.546674 3894 leader_election.cc:288] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f7ecf2c69394d2eb92172f67b98d8aa (127.23.200.195:43335), 8ba1c663c66d4941ba2bc5fb0f482053 (127.23.200.194:46497)
I20260807 20:13:12.549649 3873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.547009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:56677 (local address 127.23.200.195:43335)
0807 20:13:12.547062 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.547064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.547072 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:13:12.547100 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.547101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.547111 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.547157 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:12.547411 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.547926 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.548916 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.549327 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.549390 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.549400 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.549540 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.549576 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:13:12.549657 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.546881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:43335 (local address 127.23.200.193:56677)
0807 20:13:12.546925 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:12.546936 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:12.546980 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:12.547209 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:12.547210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:12.547212 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:12.547323 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.547325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.548006 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.548007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.548828 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.548830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.549399 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.549400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.549466 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.549470 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:13:12.549515 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 20:13:12.549557 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":32}
I20260807 20:13:12.549671 3874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.546977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:34991 (local address 127.23.200.194:46497)
0807 20:13:12.547043 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.547045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.547132 (+ 87us) server_negotiation.cc:407] Connection header received
0807 20:13:12.547171 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.547172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.547182 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.547237 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:12.547394 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.547922 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.548916 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.549337 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.549400 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.549409 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.549566 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.549604 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 20:13:12.549717 3342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.546881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:46497 (local address 127.23.200.193:34991)
0807 20:13:12.547072 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:12.547081 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:12.547125 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:12.547254 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:12.547256 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:12.547257 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:12.547330 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.547332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.547999 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.548000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.548827 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:12.548829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.549417 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.549418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:12.549506 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.549508 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:13:12.549554 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 20:13:12.549594 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":179}
I20260807 20:13:12.550038 3496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5530fa790e3340feb27dc8d90234bb2f" candidate_uuid: "c0b79210210043cd8434b1c3e065f28c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" is_pre_election: true
I20260807 20:13:12.550042 3731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5530fa790e3340feb27dc8d90234bb2f" candidate_uuid: "c0b79210210043cd8434b1c3e065f28c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" is_pre_election: true
I20260807 20:13:12.550203 3731 raft_consensus.cc:2454] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0b79210210043cd8434b1c3e065f28c in term 0.
I20260807 20:13:12.550202 3496 raft_consensus.cc:2454] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0b79210210043cd8434b1c3e065f28c in term 0.
I20260807 20:13:12.550457 3132 leader_election.cc:302] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f7ecf2c69394d2eb92172f67b98d8aa, c0b79210210043cd8434b1c3e065f28c; no voters:
I20260807 20:13:12.550630 3894 raft_consensus.cc:2781] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:13:12.550690 3894 raft_consensus.cc:482] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:13:12.550707 3894 raft_consensus.cc:3037] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:12.551396 3894 raft_consensus.cc:504] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:12.551515 3894 leader_election.cc:288] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [CANDIDATE]: Term 1 election: Requested vote from peers 6f7ecf2c69394d2eb92172f67b98d8aa (127.23.200.195:43335), 8ba1c663c66d4941ba2bc5fb0f482053 (127.23.200.194:46497)
I20260807 20:13:12.551661 3731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5530fa790e3340feb27dc8d90234bb2f" candidate_uuid: "c0b79210210043cd8434b1c3e065f28c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa"
I20260807 20:13:12.551668 3496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5530fa790e3340feb27dc8d90234bb2f" candidate_uuid: "c0b79210210043cd8434b1c3e065f28c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ba1c663c66d4941ba2bc5fb0f482053"
I20260807 20:13:12.551735 3731 raft_consensus.cc:3037] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:12.551748 3496 raft_consensus.cc:3037] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:12.552464 3731 raft_consensus.cc:2454] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0b79210210043cd8434b1c3e065f28c in term 1.
I20260807 20:13:12.552479 3496 raft_consensus.cc:2454] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0b79210210043cd8434b1c3e065f28c in term 1.
I20260807 20:13:12.552632 3134 leader_election.cc:302] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ba1c663c66d4941ba2bc5fb0f482053, c0b79210210043cd8434b1c3e065f28c; no voters:
I20260807 20:13:12.552736 3894 raft_consensus.cc:2781] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:12.552918 3894 raft_consensus.cc:686] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [term 1 LEADER]: Becoming Leader. State: Replica: c0b79210210043cd8434b1c3e065f28c, State: Running, Role: LEADER
I20260807 20:13:12.553011 3894 consensus_queue.cc:237] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:12.553882 1692 catalog_manager.cc:5495] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c reported cstate change: term changed from 0 to 1, leader changed from <none> to c0b79210210043cd8434b1c3e065f28c (127.23.200.193). New cstate: current_term: 1 leader_uuid: "c0b79210210043cd8434b1c3e065f28c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } health_report { overall_health: UNKNOWN } } }
I20260807 20:13:12.554471 3863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9859649d8d244f6983ca11af92fdb052" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: NON_VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: true } } }
I20260807 20:13:12.554565 3863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:12.555099 3863 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052: attempting to promote NON_VOTER cfff710829cd4890b60ca43bc5a008ed to VOTER
I20260807 20:13:12.555228 3863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } }
I20260807 20:13:12.555594 2683 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 FOLLOWER]: Refusing update from remote peer 9859649d8d244f6983ca11af92fdb052: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260807 20:13:12.555685 3030 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 1 LEARNER]: Refusing update from remote peer 9859649d8d244f6983ca11af92fdb052: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260807 20:13:12.555795 3906 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260807 20:13:12.555910 3853 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260807 20:13:12.556418 3906 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, cfff710829cd4890b60ca43bc5a008ed (127.23.200.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } } }
I20260807 20:13:12.556573 2683 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, cfff710829cd4890b60ca43bc5a008ed (127.23.200.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } } }
I20260807 20:13:12.556722 3030 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, cfff710829cd4890b60ca43bc5a008ed (127.23.200.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } } }
I20260807 20:13:12.557536 3853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9859649d8d244f6983ca11af92fdb052" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } } }
I20260807 20:13:12.557623 3853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:12.557637 3907 sys_catalog.cc:455] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9859649d8d244f6983ca11af92fdb052" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } } }
I20260807 20:13:12.557727 3907 sys_catalog.cc:458] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:13:12.557564 3863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9859649d8d244f6983ca11af92fdb052" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } } }
I20260807 20:13:12.557768 3863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:12.557860 3863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9859649d8d244f6983ca11af92fdb052" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } } }
I20260807 20:13:12.557919 3863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:12.557926 3879 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9859649d8d244f6983ca11af92fdb052" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } } }
I20260807 20:13:12.558004 3879 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:13:12.563051 3875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.561052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55748 (local address 127.23.200.193:44069)
0807 20:13:12.561109 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.561111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.561137 (+ 26us) server_negotiation.cc:407] Connection header received
0807 20:13:12.561174 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.561175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.561191 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.561242 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:13:12.561334 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.561792 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.562467 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.562643 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.562685 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.562787 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:13:12.562883 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:13:12.562951 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.562963 (+ 12us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.562980 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:13:12.564740 3731 raft_consensus.cc:1261] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa [term 1 FOLLOWER]: Refusing update from remote peer c0b79210210043cd8434b1c3e065f28c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:12.564788 3496 raft_consensus.cc:1261] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053 [term 1 FOLLOWER]: Refusing update from remote peer c0b79210210043cd8434b1c3e065f28c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:12.565063 3894 consensus_queue.cc:1035] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:12.565234 3898 consensus_queue.cc:1035] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:12.568161 3909 mvcc.cc:205] Tried to move back new op lower bound from 7316003195142672384 to 7316003195097751552. Current Snapshot: MvccSnapshot[applied={T|T < 7316003195142672384}]
I20260807 20:13:12.569191 3913 mvcc.cc:205] Tried to move back new op lower bound from 7316003195142672384 to 7316003195097751552. Current Snapshot: MvccSnapshot[applied={T|T < 7316003195142672384}]
I20260807 20:13:12.571452 1691 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:12.571547 1691 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:12.572793 1691 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9859649d8d244f6983ca11af92fdb052: Sending DeleteTablet for 3 replicas of tablet 5530fa790e3340feb27dc8d90234bb2f
I20260807 20:13:12.573168 3213 tablet_service.cc:1514] Processing DeleteTablet for tablet 5530fa790e3340feb27dc8d90234bb2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:12 UTC) from {username='oryx', principal='oryx/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:55744
I20260807 20:13:12.573166 3456 tablet_service.cc:1514] Processing DeleteTablet for tablet 5530fa790e3340feb27dc8d90234bb2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:12 UTC) from {username='oryx', principal='oryx/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:53950
I20260807 20:13:12.573427 3928 tablet_replica.cc:330] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053: stopping tablet replica
I20260807 20:13:12.573523 3928 raft_consensus.cc:2229] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:12.573566 3699 tablet_service.cc:1514] Processing DeleteTablet for tablet 5530fa790e3340feb27dc8d90234bb2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:12 UTC) from {username='oryx', principal='oryx/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:59866
I20260807 20:13:12.573623 3928 raft_consensus.cc:2258] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:12.573848 3931 tablet_replica.cc:330] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa: stopping tablet replica
I20260807 20:13:12.573944 3931 raft_consensus.cc:2229] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:12.574036 3931 raft_consensus.cc:2258] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:12.574038 3928 ts_tablet_manager.cc:1905] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:12.574465 3931 ts_tablet_manager.cc:1905] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:12.575188 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 1643
I20260807 20:13:12.587401 3928 ts_tablet_manager.cc:1918] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:13:12.587538 3928 log.cc:1196] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/5530fa790e3340feb27dc8d90234bb2f
I20260807 20:13:12.587944 3928 ts_tablet_manager.cc:1939] T 5530fa790e3340feb27dc8d90234bb2f P 8ba1c663c66d4941ba2bc5fb0f482053: Deleting consensus metadata
I20260807 20:13:12.589566 3930 tablet_replica.cc:330] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c: stopping tablet replica
I20260807 20:13:12.589704 3930 raft_consensus.cc:2229] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:13:12.589865 3930 raft_consensus.cc:2258] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:12.590004 3931 ts_tablet_manager.cc:1918] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:13:12.590101 3931 log.cc:1196] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/5530fa790e3340feb27dc8d90234bb2f
I20260807 20:13:12.590394 3931 ts_tablet_manager.cc:1939] T 5530fa790e3340feb27dc8d90234bb2f P 6f7ecf2c69394d2eb92172f67b98d8aa: Deleting consensus metadata
I20260807 20:13:12.591202 3930 ts_tablet_manager.cc:1905] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:12.592831 3930 ts_tablet_manager.cc:1918] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:13:12.592909 3930 log.cc:1196] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-0/wal/wals/5530fa790e3340feb27dc8d90234bb2f
I20260807 20:13:12.593206 3930 ts_tablet_manager.cc:1939] T 5530fa790e3340feb27dc8d90234bb2f P c0b79210210043cd8434b1c3e065f28c: Deleting consensus metadata
W20260807 20:13:12.593637 3371 connection.cc:535] server connection from 127.0.0.1:53950 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 20:13:12.593914 3612 connection.cc:535] server connection from 127.0.0.1:59866 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 20:13:12.594020 3132 connection.cc:535] server connection from 127.0.0.1:55744 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 20:13:12.602698 3843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.595434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55658 (local address 127.23.200.253:42419)
0807 20:13:12.595478 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.595481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.595718 (+ 237us) server_negotiation.cc:407] Connection header received
0807 20:13:12.595757 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.595758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.595775 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.595831 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:12.596444 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.597276 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.598183 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.598392 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.598463 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.599961 (+ 1498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:12.599962 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:12.599963 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:12.599971 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:12.601501 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.601828 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.601829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.601829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.601886 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.602054 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.602055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.602055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.602191 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:12.602283 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.602496 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.602595 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:13:12.603245 3850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:12.595595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41794 (local address 127.23.200.252:34687)
0807 20:13:12.595644 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:12.595646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:12.595869 (+ 223us) server_negotiation.cc:407] Connection header received
0807 20:13:12.595894 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:12.595895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:12.595905 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:12.595956 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:12.596507 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.597004 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.598000 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:12.598326 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:12.598408 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:12.600260 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:12.600261 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:12.600261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:12.600269 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:12.601747 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.602121 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.602122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.602122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.602178 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:12.602363 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:12.602364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:12.602364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:12.602495 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:12.602594 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:12.603062 (+ 468us) server_negotiation.cc:299] Negotiation successful
0807 20:13:12.603146 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:13:13.569555 3950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:13.569288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.194:60033)
0807 20:13:13.569407 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:13.569494 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":48,"threads_started":1}
W20260807 20:13:13.569796 3570 heartbeater.cc:644] Failed to heartbeat to 127.23.200.254:37725 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
I20260807 20:13:13.570108 3951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:13.569500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.193:42853)
0807 20:13:13.570000 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:13.570067 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":72,"threads_started":1}
W20260807 20:13:13.570292 3336 heartbeater.cc:644] Failed to heartbeat to 127.23.200.254:37725 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
I20260807 20:13:13.570359 3949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:13.569183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.195:40963)
0807 20:13:13.569408 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:13.570309 (+ 901us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":78,"threads_started":1}
W20260807 20:13:13.570533 3822 heartbeater.cc:644] Failed to heartbeat to 127.23.200.254:37725 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
I20260807 20:13:14.073334 3952 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9859649d8d244f6983ca11af92fdb052)
I20260807 20:13:14.073503 3952 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } }
I20260807 20:13:14.073639 3955 raft_consensus.cc:482] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9859649d8d244f6983ca11af92fdb052)
I20260807 20:13:14.073787 3952 leader_election.cc:288] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9859649d8d244f6983ca11af92fdb052 (127.23.200.254:37725), d80a1bea047e480394df4430b235c8aa (127.23.200.253:42419)
I20260807 20:13:14.073781 3955 raft_consensus.cc:504] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } }
I20260807 20:13:14.074040 3955 leader_election.cc:288] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9859649d8d244f6983ca11af92fdb052 (127.23.200.254:37725), cfff710829cd4890b60ca43bc5a008ed (127.23.200.252:34687)
I20260807 20:13:14.074189 3957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:14.073925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:53912)
0807 20:13:14.074070 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:14.074143 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":48,"threads_started":1}
I20260807 20:13:14.074393 3967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:14.074154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:53926)
0807 20:13:14.074309 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:14.074354 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":45,"threads_started":1}
W20260807 20:13:14.074424 2980 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
W20260807 20:13:14.074486 2638 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
I20260807 20:13:14.074863 3967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:14.074777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:53946)
0807 20:13:14.074812 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:14.074838 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260807 20:13:14.074959 2638 leader_election.cc:334] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9859649d8d244f6983ca11af92fdb052 (127.23.200.254:37725): Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
I20260807 20:13:14.075076 3957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:14.074725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:53934)
0807 20:13:14.074980 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:14.075047 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":61,"threads_started":1}
W20260807 20:13:14.075150 2980 leader_election.cc:334] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9859649d8d244f6983ca11af92fdb052 (127.23.200.254:37725): Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
I20260807 20:13:14.077950 3959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:14.073960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:42419 (local address 127.0.0.1:55674)
0807 20:13:14.074194 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:14.074205 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:14.074277 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:14.074479 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:14.074480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:14.074482 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:14.074542 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:14.074544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:14.075492 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:14.075494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:14.076788 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:14.076792 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:14.077628 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:14.077629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:14.077725 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:14.077729 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:13:14.077790 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 20:13:14.077835 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":69,"threads_started":1}
I20260807 20:13:14.077966 3964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:14.074032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55674 (local address 127.23.200.253:42419)
0807 20:13:14.074272 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:14.074274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:14.074284 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:13:14.074363 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:14.074364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:14.074375 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:14.074425 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:14.074615 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:14.075383 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:14.076895 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:14.077539 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:14.077620 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:14.077629 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:14.077821 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 20:13:14.077870 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":45,"threads_started":1}
I20260807 20:13:14.077960 3976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:14.074574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41796 (local address 127.23.200.252:34687)
0807 20:13:14.074685 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:14.074687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:14.074694 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:13:14.074711 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:14.074711 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:14.074720 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:14.074908 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:14.074935 (+ 27us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:14.075575 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:14.076973 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:14.077539 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:14.077620 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:14.077629 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:14.077828 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 20:13:14.077876 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":73,"threads_started":1}
I20260807 20:13:14.077972 3971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:14.074226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:34687 (local address 127.0.0.1:41796)
0807 20:13:14.074431 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:14.074441 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:14.074479 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:14.074817 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:14.074818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:14.074821 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:14.074856 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:14.074858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:14.075663 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:14.075664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:14.076887 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:14.076891 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:14.077628 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:14.077629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:14.077717 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:14.077723 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 20:13:14.077790 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 20:13:14.077835 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":44,"threads_started":1}
I20260807 20:13:14.078229 2683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cfff710829cd4890b60ca43bc5a008ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d80a1bea047e480394df4430b235c8aa" is_pre_election: true
I20260807 20:13:14.078330 2683 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfff710829cd4890b60ca43bc5a008ed in term 1.
I20260807 20:13:14.078348 3030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d80a1bea047e480394df4430b235c8aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cfff710829cd4890b60ca43bc5a008ed" is_pre_election: true
I20260807 20:13:14.078431 3030 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d80a1bea047e480394df4430b235c8aa in term 1.
I20260807 20:13:14.078490 2986 leader_election.cc:302] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cfff710829cd4890b60ca43bc5a008ed, d80a1bea047e480394df4430b235c8aa; no voters: 9859649d8d244f6983ca11af92fdb052
I20260807 20:13:14.078559 2641 leader_election.cc:302] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cfff710829cd4890b60ca43bc5a008ed, d80a1bea047e480394df4430b235c8aa; no voters: 9859649d8d244f6983ca11af92fdb052
I20260807 20:13:14.078595 3952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 20:13:14.078646 3952 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9859649d8d244f6983ca11af92fdb052)
I20260807 20:13:14.078662 3952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 1 FOLLOWER]: Advancing to term 2
I20260807 20:13:14.078644 3955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 20:13:14.078677 3955 raft_consensus.cc:482] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9859649d8d244f6983ca11af92fdb052)
I20260807 20:13:14.078696 3955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 1 FOLLOWER]: Advancing to term 2
I20260807 20:13:14.079335 3952 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } }
I20260807 20:13:14.079375 3955 raft_consensus.cc:504] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } }
I20260807 20:13:14.079445 3952 leader_election.cc:288] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [CANDIDATE]: Term 2 election: Requested vote from peers 9859649d8d244f6983ca11af92fdb052 (127.23.200.254:37725), d80a1bea047e480394df4430b235c8aa (127.23.200.253:42419)
I20260807 20:13:14.079481 3955 leader_election.cc:288] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [CANDIDATE]: Term 2 election: Requested vote from peers 9859649d8d244f6983ca11af92fdb052 (127.23.200.254:37725), cfff710829cd4890b60ca43bc5a008ed (127.23.200.252:34687)
I20260807 20:13:14.079627 3959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:14.079545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:53952)
0807 20:13:14.079580 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:14.079607 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260807 20:13:14.079715 3971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:14.079627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:53954)
0807 20:13:14.079661 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:14.079689 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260807 20:13:14.079756 2683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cfff710829cd4890b60ca43bc5a008ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d80a1bea047e480394df4430b235c8aa"
I20260807 20:13:14.079833 2683 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cfff710829cd4890b60ca43bc5a008ed in current term 2: Already voted for candidate d80a1bea047e480394df4430b235c8aa in this term.
I20260807 20:13:14.079921 3030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d80a1bea047e480394df4430b235c8aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cfff710829cd4890b60ca43bc5a008ed"
I20260807 20:13:14.079999 3030 raft_consensus.cc:2379] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d80a1bea047e480394df4430b235c8aa in current term 2: Already voted for candidate cfff710829cd4890b60ca43bc5a008ed in this term.
I20260807 20:13:14.080039 3959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:14.079838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:53968)
0807 20:13:14.079965 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:14.079994 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260807 20:13:14.080137 3971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:14.079896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:53980)
0807 20:13:14.080092 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:14.080117 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
W20260807 20:13:14.080128 2980 leader_election.cc:334] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9859649d8d244f6983ca11af92fdb052 (127.23.200.254:37725): Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
I20260807 20:13:14.080173 2980 leader_election.cc:302] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cfff710829cd4890b60ca43bc5a008ed; no voters: 9859649d8d244f6983ca11af92fdb052, d80a1bea047e480394df4430b235c8aa
I20260807 20:13:14.080227 3952 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260807 20:13:14.080274 2638 leader_election.cc:334] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9859649d8d244f6983ca11af92fdb052 (127.23.200.254:37725): Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
I20260807 20:13:14.080320 2638 leader_election.cc:302] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d80a1bea047e480394df4430b235c8aa; no voters: 9859649d8d244f6983ca11af92fdb052, cfff710829cd4890b60ca43bc5a008ed
I20260807 20:13:14.080637 3955 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 20:13:14.570621 3981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:14.570240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.194:34889)
0807 20:13:14.570410 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:14.570559 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260807 20:13:14.571082 3984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:14.570902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.195:58589)
0807 20:13:14.571008 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:14.571039 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":50,"threads_started":1}
I20260807 20:13:14.571050 3983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:14.570836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.193:37675)
0807 20:13:14.570970 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:14.571002 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260807 20:13:15.571573 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.571333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.194:59783)
0807 20:13:15.571473 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.571514 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260807 20:13:15.571839 3986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.571478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.193:36497)
0807 20:13:15.571752 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.571792 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":69,"threads_started":1}
I20260807 20:13:15.572088 3990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.571680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.195:35877)
0807 20:13:15.572005 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.572041 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":53,"threads_started":1}
I20260807 20:13:15.733175 3993 raft_consensus.cc:482] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:13:15.733285 3993 raft_consensus.cc:504] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } }
I20260807 20:13:15.733465 3993 leader_election.cc:288] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9859649d8d244f6983ca11af92fdb052 (127.23.200.254:37725), cfff710829cd4890b60ca43bc5a008ed (127.23.200.252:34687)
I20260807 20:13:15.733717 3030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d80a1bea047e480394df4430b235c8aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cfff710829cd4890b60ca43bc5a008ed" is_pre_election: true
I20260807 20:13:15.733783 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.733584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:54234)
0807 20:13:15.733707 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.733744 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
I20260807 20:13:15.733807 3030 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d80a1bea047e480394df4430b235c8aa in term 2.
I20260807 20:13:15.733966 2641 leader_election.cc:302] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cfff710829cd4890b60ca43bc5a008ed, d80a1bea047e480394df4430b235c8aa; no voters:
I20260807 20:13:15.734071 3993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 20:13:15.734120 3993 raft_consensus.cc:482] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:13:15.734138 3993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 2 FOLLOWER]: Advancing to term 3
I20260807 20:13:15.734242 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.734155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:54250)
0807 20:13:15.734198 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.734219 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 20:13:15.734313 2638 leader_election.cc:334] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9859649d8d244f6983ca11af92fdb052 (127.23.200.254:37725): Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
I20260807 20:13:15.734917 3993 raft_consensus.cc:504] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } }
I20260807 20:13:15.735036 3993 leader_election.cc:288] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [CANDIDATE]: Term 3 election: Requested vote from peers 9859649d8d244f6983ca11af92fdb052 (127.23.200.254:37725), cfff710829cd4890b60ca43bc5a008ed (127.23.200.252:34687)
I20260807 20:13:15.735234 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.735127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:54254)
0807 20:13:15.735177 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.735210 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 20:13:15.735210 3030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d80a1bea047e480394df4430b235c8aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cfff710829cd4890b60ca43bc5a008ed"
I20260807 20:13:15.735291 3030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 2 FOLLOWER]: Advancing to term 3
I20260807 20:13:15.735546 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.735444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:54266)
0807 20:13:15.735494 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.735524 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 20:13:15.735626 2638 leader_election.cc:334] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9859649d8d244f6983ca11af92fdb052 (127.23.200.254:37725): Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
I20260807 20:13:15.736081 3030 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d80a1bea047e480394df4430b235c8aa in term 3.
I20260807 20:13:15.736255 2641 leader_election.cc:302] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cfff710829cd4890b60ca43bc5a008ed, d80a1bea047e480394df4430b235c8aa; no voters: 9859649d8d244f6983ca11af92fdb052
I20260807 20:13:15.736343 3993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 3 FOLLOWER]: Leader election won for term 3
I20260807 20:13:15.736497 3993 raft_consensus.cc:686] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [term 3 LEADER]: Becoming Leader. State: Replica: d80a1bea047e480394df4430b235c8aa, State: Running, Role: LEADER
I20260807 20:13:15.736593 3993 consensus_queue.cc:237] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } }
I20260807 20:13:15.736931 3996 sys_catalog.cc:455] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader d80a1bea047e480394df4430b235c8aa. Latest consensus state: current_term: 3 leader_uuid: "d80a1bea047e480394df4430b235c8aa" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } } }
I20260807 20:13:15.737026 3996 sys_catalog.cc:458] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:15.737232 3998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:13:15.737718 3998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:15.737790 3998 catalog_manager.cc:1259] Loaded cluster ID: 09fc8a617b444902b2c24ba4c910146c
I20260807 20:13:15.737818 3998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:13:15.738025 3998 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:15.738108 3998 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: Loaded TSK: 0
I20260807 20:13:15.738252 3998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:13:15.763224 2667 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:13:15.764858 2667 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:13:15.766000 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.765868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:54292)
0807 20:13:15.765918 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.765951 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 20:13:15.766022 3030 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [term 3 FOLLOWER]: Refusing update from remote peer d80a1bea047e480394df4430b235c8aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260807 20:13:15.766270 3993 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 20:13:15.766407 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.766318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.0.0.1:54296)
0807 20:13:15.766355 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.766378 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260807 20:13:15.766474 2638 consensus_peers.cc:487] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa -> Peer 9859649d8d244f6983ca11af92fdb052 (127.23.200.254:37725): Couldn't send request to peer 9859649d8d244f6983ca11af92fdb052. Status: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 20:13:15.766836 4005 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader d80a1bea047e480394df4430b235c8aa. Latest consensus state: current_term: 3 leader_uuid: "d80a1bea047e480394df4430b235c8aa" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } } }
I20260807 20:13:15.766932 4005 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:13:15.767428 3996 sys_catalog.cc:455] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "d80a1bea047e480394df4430b235c8aa" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } } }
I20260807 20:13:15.767513 3996 sys_catalog.cc:458] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:15.767620 4005 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "d80a1bea047e480394df4430b235c8aa" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "9859649d8d244f6983ca11af92fdb052" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37725 } } peers { permanent_uuid: "d80a1bea047e480394df4430b235c8aa" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 42419 } attrs { promote: false } } peers { permanent_uuid: "cfff710829cd4890b60ca43bc5a008ed" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 34687 } attrs { promote: false } } }
I20260807 20:13:15.767696 4005 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfff710829cd4890b60ca43bc5a008ed [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:13:15.773034 4017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.769797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53964 (local address 127.23.200.194:46497)
0807 20:13:15.770244 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:15.770246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:15.770252 (+ 6us) server_negotiation.cc:407] Connection header received
0807 20:13:15.770271 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:15.770272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:15.770281 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:15.770351 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:15.770469 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:15.771180 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:15.772103 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:15.772655 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:15.772728 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:15.772740 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:15.772899 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 20:13:15.772944 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":51,"threads_started":1}
I20260807 20:13:15.773020 4013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.769553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:46497 (local address 127.0.0.1:53964)
0807 20:13:15.769910 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.769920 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:15.769962 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:15.770352 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:15.770353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:15.770354 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:15.770386 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:15.770387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:15.771251 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:15.771252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:15.771996 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:15.771998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:15.772773 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:15.772774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:15.772829 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:15.772833 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:13:15.772877 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 20:13:15.772921 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":75,"threads_started":1}
I20260807 20:13:15.773121 4015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.769610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55758 (local address 127.23.200.193:44069)
0807 20:13:15.769735 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:15.769738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:15.769747 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:15.769775 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:15.769776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:15.769788 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:15.769932 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:15.770136 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:15.770765 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:15.772029 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:15.772687 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:15.772752 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:15.772761 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:15.772973 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 20:13:15.773028 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
I20260807 20:13:15.773119 4018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.769792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59878 (local address 127.23.200.195:43335)
0807 20:13:15.770080 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:15.770082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:15.770090 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:13:15.770113 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:15.770113 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:15.770124 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:15.770183 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:15.770312 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:15.771049 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:15.772339 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:15.772706 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:15.772767 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:15.772775 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:15.772972 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 20:13:15.773013 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":58,"threads_started":1}
I20260807 20:13:15.773343 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.769493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:44069 (local address 127.0.0.1:55758)
0807 20:13:15.769539 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.769550 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:15.769600 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:15.769960 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:15.769961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:15.769963 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:15.770016 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:15.770018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:15.770837 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:15.770838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:15.771930 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:15.771933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:15.772771 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:15.772771 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:15.772834 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:15.772838 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:13:15.772886 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 20:13:15.772933 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":1}
I20260807 20:13:15.773343 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.769506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:43335 (local address 127.0.0.1:59878)
0807 20:13:15.769669 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.769677 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:15.769713 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:15.770189 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:15.770191 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:15.770193 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:15.770235 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:15.770237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:15.771132 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:15.771133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:15.772240 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:15.772242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:15.772803 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:15.772804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:15.772865 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:15.772868 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:13:15.772907 (+ 39us) client_negotiation.cc:240] Negotiation successful
0807 20:13:15.772937 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260807 20:13:15.773594 3699 tablet_service.cc:1467] Processing CreateTablet for tablet 3f6cda063a38432d99c2640dd64d21cd (DEFAULT_TABLE table=test-table [id=f3a451b0555e4dd38458b5c1bbf883e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:15.773689 3213 tablet_service.cc:1467] Processing CreateTablet for tablet 3f6cda063a38432d99c2640dd64d21cd (DEFAULT_TABLE table=test-table [id=f3a451b0555e4dd38458b5c1bbf883e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:15.773754 3699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f6cda063a38432d99c2640dd64d21cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:15.773794 3456 tablet_service.cc:1467] Processing CreateTablet for tablet 3f6cda063a38432d99c2640dd64d21cd (DEFAULT_TABLE table=test-table [id=f3a451b0555e4dd38458b5c1bbf883e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:15.773828 3213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f6cda063a38432d99c2640dd64d21cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:15.773892 3456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f6cda063a38432d99c2640dd64d21cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:15.775027 4019 tablet_bootstrap.cc:492] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa: Bootstrap starting.
I20260807 20:13:15.775255 4023 tablet_bootstrap.cc:492] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053: Bootstrap starting.
I20260807 20:13:15.775532 4019 tablet_bootstrap.cc:654] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:15.775576 4023 tablet_bootstrap.cc:654] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:15.775864 4022 tablet_bootstrap.cc:492] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c: Bootstrap starting.
I20260807 20:13:15.776156 4023 tablet_bootstrap.cc:492] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053: No bootstrap required, opened a new log
I20260807 20:13:15.776189 4022 tablet_bootstrap.cc:654] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:15.776211 4023 ts_tablet_manager.cc:1397] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:15.776348 4023 raft_consensus.cc:348] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:15.776433 4023 raft_consensus.cc:374] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:15.776453 4023 raft_consensus.cc:729] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ba1c663c66d4941ba2bc5fb0f482053, State: Initialized, Role: FOLLOWER
I20260807 20:13:15.776500 4023 consensus_queue.cc:260] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:15.776676 4023 ts_tablet_manager.cc:1428] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:15.776970 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.776875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.194:49097)
0807 20:13:15.776910 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.776940 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260807 20:13:15.777292 3573 heartbeater.cc:503] Master 127.23.200.253:42419 was elected leader, sending a full tablet report...
I20260807 20:13:15.777451 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.777357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.194:37665)
0807 20:13:15.777400 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.777425 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 20:13:15.786084 4019 tablet_bootstrap.cc:492] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa: No bootstrap required, opened a new log
I20260807 20:13:15.786084 4022 tablet_bootstrap.cc:492] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c: No bootstrap required, opened a new log
I20260807 20:13:15.786161 4022 ts_tablet_manager.cc:1397] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260807 20:13:15.786161 4019 ts_tablet_manager.cc:1397] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260807 20:13:15.786300 4022 raft_consensus.cc:348] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:15.786379 4022 raft_consensus.cc:374] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:15.786343 4019 raft_consensus.cc:348] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:15.786415 4022 raft_consensus.cc:729] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0b79210210043cd8434b1c3e065f28c, State: Initialized, Role: FOLLOWER
I20260807 20:13:15.786415 4019 raft_consensus.cc:374] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:15.786440 4019 raft_consensus.cc:729] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7ecf2c69394d2eb92172f67b98d8aa, State: Initialized, Role: FOLLOWER
I20260807 20:13:15.786485 4019 consensus_queue.cc:260] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:15.786492 4022 consensus_queue.cc:260] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:15.786687 4022 ts_tablet_manager.cc:1428] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:15.786690 4019 ts_tablet_manager.cc:1428] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:15.787138 3990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.787029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.195:39057)
0807 20:13:15.787074 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.787102 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 20:13:15.787248 3337 heartbeater.cc:503] Master 127.23.200.253:42419 was elected leader, sending a full tablet report...
I20260807 20:13:15.787622 3823 heartbeater.cc:503] Master 127.23.200.253:42419 was elected leader, sending a full tablet report...
I20260807 20:13:15.787639 3986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.787514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.193:51021)
0807 20:13:15.787574 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.787611 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 20:13:15.787763 3990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.787514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.195:44503)
0807 20:13:15.787714 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.787742 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
W20260807 20:13:15.838974 3577 tablet.cc:2369] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:13:15.894076 4032 raft_consensus.cc:482] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:13:15.894172 4032 raft_consensus.cc:504] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:15.894357 4032 leader_election.cc:288] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f7ecf2c69394d2eb92172f67b98d8aa (127.23.200.195:43335), 8ba1c663c66d4941ba2bc5fb0f482053 (127.23.200.194:46497)
I20260807 20:13:15.894593 3496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6cda063a38432d99c2640dd64d21cd" candidate_uuid: "c0b79210210043cd8434b1c3e065f28c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" is_pre_election: true
I20260807 20:13:15.894598 3731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6cda063a38432d99c2640dd64d21cd" candidate_uuid: "c0b79210210043cd8434b1c3e065f28c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" is_pre_election: true
I20260807 20:13:15.894683 3496 raft_consensus.cc:2454] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0b79210210043cd8434b1c3e065f28c in term 0.
I20260807 20:13:15.894691 3731 raft_consensus.cc:2454] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0b79210210043cd8434b1c3e065f28c in term 0.
I20260807 20:13:15.894824 3134 leader_election.cc:302] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ba1c663c66d4941ba2bc5fb0f482053, c0b79210210043cd8434b1c3e065f28c; no voters:
I20260807 20:13:15.894959 4032 raft_consensus.cc:2781] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:13:15.895005 4032 raft_consensus.cc:482] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:13:15.895023 4032 raft_consensus.cc:3037] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:15.895788 4032 raft_consensus.cc:504] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:15.895915 4032 leader_election.cc:288] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [CANDIDATE]: Term 1 election: Requested vote from peers 6f7ecf2c69394d2eb92172f67b98d8aa (127.23.200.195:43335), 8ba1c663c66d4941ba2bc5fb0f482053 (127.23.200.194:46497)
I20260807 20:13:15.896073 3731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6cda063a38432d99c2640dd64d21cd" candidate_uuid: "c0b79210210043cd8434b1c3e065f28c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa"
I20260807 20:13:15.896138 3731 raft_consensus.cc:3037] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:15.896126 3496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6cda063a38432d99c2640dd64d21cd" candidate_uuid: "c0b79210210043cd8434b1c3e065f28c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ba1c663c66d4941ba2bc5fb0f482053"
I20260807 20:13:15.896201 3496 raft_consensus.cc:3037] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:15.896728 3731 raft_consensus.cc:2454] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0b79210210043cd8434b1c3e065f28c in term 1.
I20260807 20:13:15.896812 3496 raft_consensus.cc:2454] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0b79210210043cd8434b1c3e065f28c in term 1.
I20260807 20:13:15.896898 3132 leader_election.cc:302] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f7ecf2c69394d2eb92172f67b98d8aa, c0b79210210043cd8434b1c3e065f28c; no voters:
I20260807 20:13:15.896983 4032 raft_consensus.cc:2781] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:15.897145 4032 raft_consensus.cc:686] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [term 1 LEADER]: Becoming Leader. State: Replica: c0b79210210043cd8434b1c3e065f28c, State: Running, Role: LEADER
I20260807 20:13:15.897197 4032 consensus_queue.cc:237] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } }
I20260807 20:13:15.897461 3986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.897343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.193:39493)
0807 20:13:15.897389 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.897430 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260807 20:13:15.897979 2667 catalog_manager.cc:5495] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c reported cstate change: term changed from 0 to 1, leader changed from <none> to c0b79210210043cd8434b1c3e065f28c (127.23.200.193). New cstate: current_term: 1 leader_uuid: "c0b79210210043cd8434b1c3e065f28c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b79210210043cd8434b1c3e065f28c" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 } health_report { overall_health: UNKNOWN } } }
I20260807 20:13:15.917627 4015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.915418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55768 (local address 127.23.200.193:44069)
0807 20:13:15.915464 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:15.915465 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:15.915474 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:15.915542 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:15.915543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:15.915555 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:15.915600 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:13:15.915718 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:15.916190 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:15.916954 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:15.917189 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:15.917256 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:15.917392 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:13:15.917429 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:13:15.917475 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:15.917528 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 20:13:15.917551 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:13:15.918447 3496 raft_consensus.cc:1261] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053 [term 1 FOLLOWER]: Refusing update from remote peer c0b79210210043cd8434b1c3e065f28c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:15.918607 3731 raft_consensus.cc:1261] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa [term 1 FOLLOWER]: Refusing update from remote peer c0b79210210043cd8434b1c3e065f28c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:15.918699 4033 consensus_queue.cc:1035] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ba1c663c66d4941ba2bc5fb0f482053" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:15.918853 4033 consensus_queue.cc:1035] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f7ecf2c69394d2eb92172f67b98d8aa" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 43335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:15.919656 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.919338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.194:59655)
0807 20:13:15.919584 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.919623 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229}
I20260807 20:13:15.920190 3990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.920094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.195:38271)
0807 20:13:15.920134 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.920160 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 20:13:15.920969 3990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.920818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.195:34209)
0807 20:13:15.920915 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.920943 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 20:13:15.921363 3986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.921281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.193:36297)
0807 20:13:15.921307 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.921334 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
I20260807 20:13:15.921526 4044 mvcc.cc:205] Tried to move back new op lower bound from 7316003208880410624 to 7316003208795619328. Current Snapshot: MvccSnapshot[applied={T|T < 7316003208880410624}]
I20260807 20:13:15.921802 3986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.921681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.193:50919)
0807 20:13:15.921742 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.921776 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 20:13:15.922231 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.922145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37725 (local address 127.23.200.194:60581)
0807 20:13:15.922175 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.922201 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:37725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260807 20:13:15.922832 2667 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:15.922937 2667 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:15.924172 2667 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d80a1bea047e480394df4430b235c8aa: Sending DeleteTablet for 3 replicas of tablet 3f6cda063a38432d99c2640dd64d21cd
I20260807 20:13:15.924448 3213 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f6cda063a38432d99c2640dd64d21cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:15 UTC) from {username='oryx', principal='oryx/127.23.200.253@KRBTEST.COM'} at 127.0.0.1:55758
I20260807 20:13:15.924453 3699 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f6cda063a38432d99c2640dd64d21cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:15 UTC) from {username='oryx', principal='oryx/127.23.200.253@KRBTEST.COM'} at 127.0.0.1:59878
I20260807 20:13:15.924479 3456 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f6cda063a38432d99c2640dd64d21cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:15 UTC) from {username='oryx', principal='oryx/127.23.200.253@KRBTEST.COM'} at 127.0.0.1:53964
I20260807 20:13:15.924656 4058 tablet_replica.cc:330] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c: stopping tablet replica
I20260807 20:13:15.924712 4058 raft_consensus.cc:2229] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:13:15.924772 4058 raft_consensus.cc:2258] T 3f6cda063a38432d99c2640dd64d21cd P c0b79210210043cd8434b1c3e065f28c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:15.924856 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 3104
I20260807 20:13:15.932235 4059 tablet_replica.cc:330] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053: stopping tablet replica
I20260807 20:13:15.932314 4059 raft_consensus.cc:2229] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:15.932358 4059 raft_consensus.cc:2258] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:15.936244 4059 ts_tablet_manager.cc:1905] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:15.939841 4059 ts_tablet_manager.cc:1918] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:13:15.939901 4059 log.cc:1196] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/3f6cda063a38432d99c2640dd64d21cd
I20260807 20:13:15.940061 4059 ts_tablet_manager.cc:1939] T 3f6cda063a38432d99c2640dd64d21cd P 8ba1c663c66d4941ba2bc5fb0f482053: Deleting consensus metadata
I20260807 20:13:15.940367 2639 catalog_manager.cc:4841] TS 8ba1c663c66d4941ba2bc5fb0f482053 (127.23.200.194:46497): tablet 3f6cda063a38432d99c2640dd64d21cd (table test-table [id=f3a451b0555e4dd38458b5c1bbf883e6]) successfully deleted
I20260807 20:13:15.924793 4057 tablet_replica.cc:330] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa: stopping tablet replica
I20260807 20:13:15.946085 4057 raft_consensus.cc:2229] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:15.946178 4057 raft_consensus.cc:2258] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:15.947582 4057 ts_tablet_manager.cc:1905] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:15.949705 4057 ts_tablet_manager.cc:1918] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:13:15.949779 4057 log.cc:1196] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/3f6cda063a38432d99c2640dd64d21cd
I20260807 20:13:15.949998 4057 ts_tablet_manager.cc:1939] T 3f6cda063a38432d99c2640dd64d21cd P 6f7ecf2c69394d2eb92172f67b98d8aa: Deleting consensus metadata
I20260807 20:13:15.950353 2638 catalog_manager.cc:4841] TS 6f7ecf2c69394d2eb92172f67b98d8aa (127.23.200.195:43335): tablet 3f6cda063a38432d99c2640dd64d21cd (table test-table [id=f3a451b0555e4dd38458b5c1bbf883e6]) successfully deleted
I20260807 20:13:15.954605 4014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:15.954488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:44069 (local address 127.0.0.1:55782)
0807 20:13:15.954544 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:15.954570 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:44069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 20:13:15.954641 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 3345
W20260807 20:13:15.954803 2644 catalog_manager.cc:4584] TS c0b79210210043cd8434b1c3e065f28c (127.23.200.193:44069): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f6cda063a38432d99c2640dd64d21cd: Network error: Client connection negotiation failed: client connection to 127.23.200.193:44069: connect: Connection refused (error 111)
I20260807 20:13:15.970078 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 3585
I20260807 20:13:15.977687 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 2448
I20260807 20:13:15.987933 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 2772
2026-08-07T20:13:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6489 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:13:16 dist-test-slave-2j7r krb5kdc[4063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:13:16 dist-test-slave-2j7r krb5kdc[4063](info): set up 2 sockets
Aug 07 20:13:16 dist-test-slave-2j7r krb5kdc[4063](info): commencing operation
krb5kdc: starting...
W20260807 20:13:17.157940 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:13:17 dist-test-slave-2j7r krb5kdc[4063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:13:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.254.keytab.
Entry for principal oryx/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.254.keytab.
I20260807 20:13:17.283788 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:42397
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.254.keytab
--principal=oryx/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:34223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:42397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:17.367126 4085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:17.367342 4085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:17.367365 4085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:17.368489 4085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:17.368522 4085 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 20:13:17.368534 4085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:17.368546 4085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:17.368556 4085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:17.368569 4085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:13:17.370172 4085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:42397
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.254.keytab
--principal=oryx/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:42397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:17.370425 4085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:17.370647 4085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:17.375679 4100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:17.375712 4096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:17.375710 4095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:17.375746 4085 server_base.cc:1034] running on GCE node
I20260807 20:13:17.376086 4085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:17.376339 4085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:17.377482 4085 hybrid_clock.cc:648] HybridClock initialized: now 1786133597377464 us; error 33 us; skew 500 ppm
Aug 07 20:13:17 dist-test-slave-2j7r krb5kdc[4063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133597, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:17.379910 4085 init.cc:377] Logged in from keytab as oryx/127.23.200.254@KRBTEST.COM (short username oryx)
I20260807 20:13:17.380375 4085 webserver.cc:460] Webserver started at http://127.23.200.254:37419/ using document root <none> and password file <none>
I20260807 20:13:17.380610 4085 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:17.380656 4085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:17.380784 4085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:17.381891 4085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "238f29fdf6514a62925344cbe8453b0a"
format_stamp: "Formatted at 2026-08-07 20:13:17 on dist-test-slave-2j7r"
server_key: "ef1e01fc8dae8c15426082161aac2c65"
server_key_iv: "86a2529d98cc875b16563a89a181dd0b"
server_key_version: "encryptionkey@0"
I20260807 20:13:17.382233 4085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "238f29fdf6514a62925344cbe8453b0a"
format_stamp: "Formatted at 2026-08-07 20:13:17 on dist-test-slave-2j7r"
server_key: "ef1e01fc8dae8c15426082161aac2c65"
server_key_iv: "86a2529d98cc875b16563a89a181dd0b"
server_key_version: "encryptionkey@0"
I20260807 20:13:17.383548 4085 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:13:17.384238 4114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:17.384464 4085 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:17.384554 4085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "238f29fdf6514a62925344cbe8453b0a"
format_stamp: "Formatted at 2026-08-07 20:13:17 on dist-test-slave-2j7r"
server_key: "ef1e01fc8dae8c15426082161aac2c65"
server_key_iv: "86a2529d98cc875b16563a89a181dd0b"
server_key_version: "encryptionkey@0"
I20260807 20:13:17.384613 4085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:17.398223 4085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:17.400044 4085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:17.400156 4085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:17.404492 4085 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:42397
I20260807 20:13:17.404557 4223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:42397 every 8 connection(s)
I20260807 20:13:17.404863 4085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:13:17.405591 4226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:17.407946 4226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a: Bootstrap starting.
I20260807 20:13:17.408564 4226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:17.408838 4226 log.cc:824] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:17.409245 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 4085
I20260807 20:13:17.409341 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:13:17.409566 24355 external_mini_cluster.cc:1428] Setting key c5342bd6a784a63f684aa83c3086064f
I20260807 20:13:17.409634 4226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a: No bootstrap required, opened a new log
I20260807 20:13:17.412230 4226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "238f29fdf6514a62925344cbe8453b0a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42397 } }
I20260807 20:13:17.412367 4226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:17.412388 4226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 238f29fdf6514a62925344cbe8453b0a, State: Initialized, Role: FOLLOWER
I20260807 20:13:17.412499 4226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "238f29fdf6514a62925344cbe8453b0a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42397 } }
I20260807 20:13:17.412569 4226 raft_consensus.cc:388] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:13:17.412595 4226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:13:17.412640 4226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:17.413306 4226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "238f29fdf6514a62925344cbe8453b0a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42397 } }
I20260807 20:13:17.413420 4226 leader_election.cc:302] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 238f29fdf6514a62925344cbe8453b0a; no voters:
I20260807 20:13:17.413571 4226 leader_election.cc:288] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:13:17.413614 4233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:17.413805 4233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [term 1 LEADER]: Becoming Leader. State: Replica: 238f29fdf6514a62925344cbe8453b0a, State: Running, Role: LEADER
I20260807 20:13:17.413861 4226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:17.413897 4233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "238f29fdf6514a62925344cbe8453b0a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42397 } }
I20260807 20:13:17.414280 4233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 238f29fdf6514a62925344cbe8453b0a. Latest consensus state: current_term: 1 leader_uuid: "238f29fdf6514a62925344cbe8453b0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "238f29fdf6514a62925344cbe8453b0a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42397 } } }
I20260807 20:13:17.414398 4233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:17.414290 4234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "238f29fdf6514a62925344cbe8453b0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "238f29fdf6514a62925344cbe8453b0a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42397 } } }
I20260807 20:13:17.414600 4234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:17.414608 4240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:13:17.415150 4240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:17.416801 4261 mvcc.cc:205] Tried to move back new op lower bound from 7316003215016173568 to 7316003215007883264. Current Snapshot: MvccSnapshot[applied={T|T < 7316003215016173568}]
Aug 07 20:13:17 dist-test-slave-2j7r krb5kdc[4063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM
I20260807 20:13:17.417289 4240 catalog_manager.cc:1347] Generated new cluster ID: 366630c570834c0eb8edf45baba4bf25
I20260807 20:13:17.417343 4240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:13:17.420766 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.410089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58064 (local address 127.23.200.254:42397)
0807 20:13:17.410278 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:17.410285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:17.410312 (+ 27us) server_negotiation.cc:407] Connection header received
0807 20:13:17.410873 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:17.410880 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:17.410930 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:17.411049 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:17.411818 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.412817 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.413846 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.414467 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.414555 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.417370 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:17.417374 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:17.417375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:17.417387 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:17.419228 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:17.419627 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:17.419630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:17.419631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:17.419725 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:17.419911 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:17.419911 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:17.419912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:17.420180 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:17.420314 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:17.420432 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 20:13:17.420562 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260807 20:13:17.429251 4240 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:13:17.429774 4240 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:17.440972 4240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a: Generated new TSK 0
I20260807 20:13:17.441219 4240 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.193.keytab.
Entry for principal oryx/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.193.keytab.
I20260807 20:13:17.478364 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:42397
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.193.keytab
--principal=oryx/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:34223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:17.563997 4265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:17.564219 4265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:17.564246 4265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:17.565431 4265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:17.565480 4265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:17.565502 4265 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:13:17.565521 4265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:13:17.567102 4265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.193.keytab
--principal=oryx/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.200.254:42397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:17.567438 4265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:17.567682 4265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:17.572625 4274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:17.572639 4275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:17.572745 4265 server_base.cc:1034] running on GCE node
W20260807 20:13:17.572639 4280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:17.573073 4265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:17.573330 4265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:17.574565 4265 hybrid_clock.cc:648] HybridClock initialized: now 1786133597574546 us; error 34 us; skew 500 ppm
Aug 07 20:13:17 dist-test-slave-2j7r krb5kdc[4063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133597, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:17.577163 4265 init.cc:377] Logged in from keytab as oryx/127.23.200.193@KRBTEST.COM (short username oryx)
I20260807 20:13:17.577644 4265 webserver.cc:460] Webserver started at http://127.23.200.193:43431/ using document root <none> and password file <none>
I20260807 20:13:17.577867 4265 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:17.577914 4265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:17.578033 4265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:17.579075 4265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "23c49522daf54d36a1f06d5ccd2a14ae"
format_stamp: "Formatted at 2026-08-07 20:13:17 on dist-test-slave-2j7r"
server_key: "f00274e9fb0e234ee5a679fd9577e37a"
server_key_iv: "425640d1918c34be6d28f309e1306b2e"
server_key_version: "encryptionkey@0"
I20260807 20:13:17.579417 4265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "23c49522daf54d36a1f06d5ccd2a14ae"
format_stamp: "Formatted at 2026-08-07 20:13:17 on dist-test-slave-2j7r"
server_key: "f00274e9fb0e234ee5a679fd9577e37a"
server_key_iv: "425640d1918c34be6d28f309e1306b2e"
server_key_version: "encryptionkey@0"
I20260807 20:13:17.580740 4265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:13:17.581529 4288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:17.581707 4265 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:13:17.581800 4265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "23c49522daf54d36a1f06d5ccd2a14ae"
format_stamp: "Formatted at 2026-08-07 20:13:17 on dist-test-slave-2j7r"
server_key: "f00274e9fb0e234ee5a679fd9577e37a"
server_key_iv: "425640d1918c34be6d28f309e1306b2e"
server_key_version: "encryptionkey@0"
I20260807 20:13:17.581862 4265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:17.597908 4265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:17.599735 4265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:17.599875 4265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:17.600391 4265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:17.600440 4265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:17.600471 4265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:17.600488 4265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:17.606947 4265 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:42787
I20260807 20:13:17.607084 4480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:42787 every 8 connection(s)
Aug 07 20:13:17 dist-test-slave-2j7r krb5kdc[4063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133597, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM
I20260807 20:13:17.607352 4265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:13:17.610057 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.600874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:50223 (local address 127.23.200.254:42397)
0807 20:13:17.600916 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:17.600918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:17.601198 (+ 280us) server_negotiation.cc:407] Connection header received
0807 20:13:17.601725 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:17.601726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:17.601741 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:17.601795 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:17.602626 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.603403 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.604923 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.605231 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.605287 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.607809 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:17.607810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:17.607811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:17.607820 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:17.608955 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:17.609345 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:17.609346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:17.609346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:17.609396 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:17.609566 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:17.609567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:17.609567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:17.609662 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:17.609709 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:17.609919 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 20:13:17.609991 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:13:17.610221 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.600867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42397 (local address 127.23.200.193:50223)
0807 20:13:17.601044 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:17.601113 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:17.601621 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:17.601866 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:17.601871 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:17.602249 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:17.602539 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.602546 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.603513 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.603514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.604803 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.604807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.605310 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.605311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.605463 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.605981 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:17.605993 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:17.607696 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:17.609021 (+ 1325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:17.609028 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:17.609029 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:17.609282 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:17.609434 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:17.609434 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:17.609434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:17.609524 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:17.609716 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:17.609719 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:17.609826 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 20:13:17.609930 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 20:13:17.610048 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260807 20:13:17.610690 4482 heartbeater.cc:347] Connected to a master server at 127.23.200.254:42397
I20260807 20:13:17.610770 4482 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:17.610957 4482 heartbeater.cc:511] Master 127.23.200.254:42397 requested a full tablet report, sending...
I20260807 20:13:17.611550 4143 ts_manager.cc:194] Registered new tserver with Master: 23c49522daf54d36a1f06d5ccd2a14ae (127.23.200.193:42787)
I20260807 20:13:17.612319 4143 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:50223
I20260807 20:13:17.614259 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 4265
I20260807 20:13:17.614374 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:13:17.614580 24355 external_mini_cluster.cc:1428] Setting key da285ec3d1240964cf8c53d7bf5dc950
I20260807 20:13:17.616974 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.611704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:45553 (local address 127.23.200.254:42397)
0807 20:13:17.611740 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:17.611742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:17.611808 (+ 66us) server_negotiation.cc:407] Connection header received
0807 20:13:17.611849 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:17.611850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:17.611859 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:17.611930 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:17.612180 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.612742 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.613626 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.613819 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.613875 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.614759 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:17.614761 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:17.614761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:17.614769 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:17.616028 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:17.616318 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:17.616319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:17.616319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:17.616368 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:17.616530 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:17.616531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:17.616532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:17.616630 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:17.616685 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:17.616821 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 20:13:17.616885 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:13:17.616989 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.611639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42397 (local address 127.23.200.193:45553)
0807 20:13:17.611756 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:17.611768 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:17.611816 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:17.611931 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:17.611932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:17.612081 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:17.612129 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.612131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.612826 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.612827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.613550 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.613553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.613877 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.613878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.613926 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.614311 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:17.614316 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:17.614661 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:17.616097 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:17.616098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:17.616098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:17.616258 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:17.616422 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:17.616422 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:17.616423 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:17.616481 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:17.616685 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:17.616685 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:17.616731 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 20:13:17.616789 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 20:13:17.616888 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":54,"thread_start_us":68,"threads_started":1}
WARNING: no policy specified for oryx/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.194.keytab.
Entry for principal oryx/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.194.keytab.
I20260807 20:13:17.640427 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:42397
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.194.keytab
--principal=oryx/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:34223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:17.724315 4496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:17.724536 4496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:17.724561 4496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:17.725755 4496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:17.725796 4496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:17.725816 4496 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:13:17.725833 4496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:13:17.727427 4496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.194.keytab
--principal=oryx/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.200.254:42397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:17.727638 4496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:17.727828 4496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:17.732784 4505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:17.732785 4509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:17.732790 4504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:17.733052 4496 server_base.cc:1034] running on GCE node
I20260807 20:13:17.733258 4496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:17.733479 4496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:17.734620 4496 hybrid_clock.cc:648] HybridClock initialized: now 1786133597734610 us; error 29 us; skew 500 ppm
Aug 07 20:13:17 dist-test-slave-2j7r krb5kdc[4063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133597, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:17.737020 4496 init.cc:377] Logged in from keytab as oryx/127.23.200.194@KRBTEST.COM (short username oryx)
I20260807 20:13:17.737448 4496 webserver.cc:460] Webserver started at http://127.23.200.194:40037/ using document root <none> and password file <none>
I20260807 20:13:17.737653 4496 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:17.737691 4496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:17.737785 4496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:17.738781 4496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "39e8c651e0244c8ebd2828fd27019c4d"
format_stamp: "Formatted at 2026-08-07 20:13:17 on dist-test-slave-2j7r"
server_key: "9303cbdc8c7d354e1d3e91c0d036d650"
server_key_iv: "a10cb5344e7770f038bb6b432dbd8837"
server_key_version: "encryptionkey@0"
I20260807 20:13:17.739073 4496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "39e8c651e0244c8ebd2828fd27019c4d"
format_stamp: "Formatted at 2026-08-07 20:13:17 on dist-test-slave-2j7r"
server_key: "9303cbdc8c7d354e1d3e91c0d036d650"
server_key_iv: "a10cb5344e7770f038bb6b432dbd8837"
server_key_version: "encryptionkey@0"
I20260807 20:13:17.740257 4496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:13:17.740962 4518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:17.741169 4496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:17.741272 4496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "39e8c651e0244c8ebd2828fd27019c4d"
format_stamp: "Formatted at 2026-08-07 20:13:17 on dist-test-slave-2j7r"
server_key: "9303cbdc8c7d354e1d3e91c0d036d650"
server_key_iv: "a10cb5344e7770f038bb6b432dbd8837"
server_key_version: "encryptionkey@0"
I20260807 20:13:17.741330 4496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:17.754875 4496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:17.756973 4496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:17.757115 4496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:17.757601 4496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:17.757643 4496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:17.757666 4496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:17.757678 4496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:17.764349 4496 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:33353
I20260807 20:13:17.764408 4715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:33353 every 8 connection(s)
Aug 07 20:13:17 dist-test-slave-2j7r krb5kdc[4063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133597, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM
I20260807 20:13:17.764753 4496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:13:17.766006 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 4496
I20260807 20:13:17.766108 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:13:17.766300 24355 external_mini_cluster.cc:1428] Setting key b929e1f6a6571f643714bbeafa1cfc7a
I20260807 20:13:17.767778 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.758148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:43815 (local address 127.23.200.254:42397)
0807 20:13:17.758200 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:17.758202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:17.758392 (+ 190us) server_negotiation.cc:407] Connection header received
0807 20:13:17.758895 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:17.758896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:17.758910 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:17.758966 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:17.759870 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.760640 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.762213 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.762523 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.762592 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.765186 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:17.765188 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:17.765189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:17.765198 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:17.766549 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:17.766893 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:17.766895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:17.766895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:17.766960 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:17.767177 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:17.767178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:17.767178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:17.767293 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:17.767349 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:17.767611 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 20:13:17.767690 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 20:13:17.767942 4535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.758105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42397 (local address 127.23.200.194:43815)
0807 20:13:17.758277 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:17.758313 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:17.758799 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:17.759095 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:17.759101 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:17.759469 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:17.759773 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.759780 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.760743 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.760744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.762104 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.762107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.762597 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.762598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.762742 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.763248 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:17.763269 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:17.765058 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:17.766610 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:17.766618 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:17.766620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:17.766826 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:17.767014 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:17.767015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:17.767015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:17.767122 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:17.767346 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:17.767352 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:17.767493 (+ 141us) client_negotiation.cc:769] Sending connection context
0807 20:13:17.767583 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 20:13:17.767735 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260807 20:13:17.768410 4716 heartbeater.cc:347] Connected to a master server at 127.23.200.254:42397
I20260807 20:13:17.768507 4716 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:17.768700 4716 heartbeater.cc:511] Master 127.23.200.254:42397 requested a full tablet report, sending...
I20260807 20:13:17.769116 4145 ts_manager.cc:194] Registered new tserver with Master: 39e8c651e0244c8ebd2828fd27019c4d (127.23.200.194:33353)
I20260807 20:13:17.769563 4145 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:43815
I20260807 20:13:17.775023 4731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.769470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42397 (local address 127.23.200.194:39967)
0807 20:13:17.769662 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:17.769673 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:17.769734 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:17.769840 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:17.769841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:17.770030 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:17.770084 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.770086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.770751 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.770752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.771240 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.771241 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.771647 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.771647 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.771702 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.772091 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:17.772096 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:17.772396 (+ 300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:17.774053 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:17.774054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:17.774054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:17.774207 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:17.774391 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:17.774391 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:17.774391 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:17.774464 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:17.774776 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:17.774776 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:17.774815 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 20:13:17.774866 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 20:13:17.774948 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":133,"threads_started":1}
I20260807 20:13:17.775059 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.769533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:39967 (local address 127.23.200.254:42397)
0807 20:13:17.769572 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:17.769574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:17.769725 (+ 151us) server_negotiation.cc:407] Connection header received
0807 20:13:17.769779 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:17.769780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:17.769790 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:17.769838 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:17.770158 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.770662 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.771316 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.771581 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.771648 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.772506 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:17.772507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:17.772507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:17.772515 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:17.773965 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:17.774273 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:17.774274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:17.774274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:17.774335 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:17.774515 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:17.774515 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:17.774516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:17.774708 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:17.774785 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:17.774893 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 20:13:17.774974 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for oryx/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.195.keytab.
Entry for principal oryx/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.195.keytab.
I20260807 20:13:17.795360 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:42397
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.195.keytab
--principal=oryx/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:34223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:17.877897 4733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:17.878152 4733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:17.878180 4733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:17.879345 4733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:17.879386 4733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:17.879405 4733 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:13:17.879421 4733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:13:17.880921 4733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.195.keytab
--principal=oryx/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.200.254:42397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:17.881179 4733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:17.881402 4733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:17.886646 4739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:17.886727 4733 server_base.cc:1034] running on GCE node
W20260807 20:13:17.886829 4740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:17.886648 4743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:17.887097 4733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:17.887359 4733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:17.888520 4733 hybrid_clock.cc:648] HybridClock initialized: now 1786133597888507 us; error 28 us; skew 500 ppm
Aug 07 20:13:17 dist-test-slave-2j7r krb5kdc[4063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133597, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:17.891044 4733 init.cc:377] Logged in from keytab as oryx/127.23.200.195@KRBTEST.COM (short username oryx)
I20260807 20:13:17.891501 4733 webserver.cc:460] Webserver started at http://127.23.200.195:37439/ using document root <none> and password file <none>
I20260807 20:13:17.891698 4733 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:17.891744 4733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:17.891872 4733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:17.892894 4733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "61087001c28c4214a2a97c681f874241"
format_stamp: "Formatted at 2026-08-07 20:13:17 on dist-test-slave-2j7r"
server_key: "a8e96b94324477c4365ec42c4b7355b7"
server_key_iv: "10bb477ab52d8e31c0092fd19dd1c956"
server_key_version: "encryptionkey@0"
I20260807 20:13:17.893245 4733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "61087001c28c4214a2a97c681f874241"
format_stamp: "Formatted at 2026-08-07 20:13:17 on dist-test-slave-2j7r"
server_key: "a8e96b94324477c4365ec42c4b7355b7"
server_key_iv: "10bb477ab52d8e31c0092fd19dd1c956"
server_key_version: "encryptionkey@0"
I20260807 20:13:17.894524 4733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:13:17.895184 4753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:17.895320 4733 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:13:17.895416 4733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "61087001c28c4214a2a97c681f874241"
format_stamp: "Formatted at 2026-08-07 20:13:17 on dist-test-slave-2j7r"
server_key: "a8e96b94324477c4365ec42c4b7355b7"
server_key_iv: "10bb477ab52d8e31c0092fd19dd1c956"
server_key_version: "encryptionkey@0"
I20260807 20:13:17.895475 4733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:17.906634 4733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:17.908696 4733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:17.908846 4733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:17.909456 4733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:17.909504 4733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:17.909535 4733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:17.909551 4733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 20:13:17 dist-test-slave-2j7r krb5kdc[4063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133597, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM
I20260807 20:13:17.916858 4733 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:34733
I20260807 20:13:17.916910 4942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:34733 every 8 connection(s)
I20260807 20:13:17.917320 4733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:13:17.919721 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.909857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:39725 (local address 127.23.200.254:42397)
0807 20:13:17.909924 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:17.909926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:17.910155 (+ 229us) server_negotiation.cc:407] Connection header received
0807 20:13:17.910895 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:17.910897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:17.910912 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:17.910961 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:17.911949 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.912717 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.914338 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.914672 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.914744 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.917194 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:17.917196 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:17.917196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:17.917205 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:17.918491 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:17.918856 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:17.918857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:17.918858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:17.918947 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:17.919169 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:17.919170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:17.919170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:17.919285 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:17.919340 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:17.919581 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 20:13:17.919648 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:13:17.919890 4765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.909842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42397 (local address 127.23.200.195:39725)
0807 20:13:17.910009 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:17.910058 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:17.910748 (+ 690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:17.911177 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:17.911182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:17.911549 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:17.911850 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.911858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.912832 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.912833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.914219 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.914222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.914825 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.914826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.914950 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.915465 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:17.915476 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:17.917110 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:17.918560 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:17.918566 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:17.918568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:17.918774 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:17.919006 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:17.919006 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:17.919006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:17.919107 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:17.919333 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:17.919337 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:17.919481 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 20:13:17.919564 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 20:13:17.919686 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":105,"threads_started":1}
I20260807 20:13:17.920359 4944 heartbeater.cc:347] Connected to a master server at 127.23.200.254:42397
I20260807 20:13:17.920449 4944 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:17.920620 4944 heartbeater.cc:511] Master 127.23.200.254:42397 requested a full tablet report, sending...
I20260807 20:13:17.920811 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 4733
I20260807 20:13:17.920926 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:13:17.921183 24355 external_mini_cluster.cc:1428] Setting key 82c341be186e5dee1c74ee0661597f9d
I20260807 20:13:17.921670 4143 ts_manager.cc:194] Registered new tserver with Master: 61087001c28c4214a2a97c681f874241 (127.23.200.195:34733)
I20260807 20:13:17.922057 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:13:17.922101 4143 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:39725
I20260807 20:13:17.927585 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.921464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:53815 (local address 127.23.200.254:42397)
0807 20:13:17.922204 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:17.922206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:17.922214 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:13:17.922232 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:17.922233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:17.922242 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:17.922288 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:17.922672 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.923266 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.923927 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.924182 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.924247 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.925161 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:17.925162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:17.925162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:17.925168 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:17.926343 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:17.926740 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:17.926742 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:17.926742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:17.926852 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:17.927050 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:17.927050 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:17.927051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:17.927161 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:17.927225 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:17.927422 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 20:13:17.927494 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720}
I20260807 20:13:17.927599 4951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.921426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42397 (local address 127.23.200.195:53815)
0807 20:13:17.921561 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:17.921572 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:17.921618 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:17.922297 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:17.922299 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:17.922519 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:17.922586 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.922589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.923364 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.923365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.923855 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.923857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.924254 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.924255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.924322 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.924726 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:17.924731 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:17.925074 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:17.926433 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:17.926434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:17.926435 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:17.926641 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:17.926923 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:17.926923 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:17.926924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:17.926992 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:17.927256 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:17.927258 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:17.927309 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 20:13:17.927380 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 20:13:17.927491 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
Aug 07 20:13:17 dist-test-slave-2j7r krb5kdc[4063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133597, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 20:13:17 dist-test-slave-2j7r krb5kdc[4063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:17 dist-test-slave-2j7r krb5kdc[4063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 20:13:17.944142 4231 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:17.938981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58068 (local address 127.23.200.254:42397)
0807 20:13:17.939029 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:17.939031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:17.939041 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:13:17.939066 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:17.939067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:17.939075 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:17.939126 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:17.939656 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.940145 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.940921 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.941217 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.941268 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.944068 (+ 2800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58068: BlockingRecv error: recv got EOF from 127.0.0.1:58068 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 07 20:13:17 dist-test-slave-2j7r krb5kdc[4063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133597, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM
I20260807 20:13:17.950999 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.944810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58080 (local address 127.23.200.254:42397)
0807 20:13:17.944872 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:17.944874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:17.945003 (+ 129us) server_negotiation.cc:407] Connection header received
0807 20:13:17.945107 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:17.945108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:17.945120 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:17.945173 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:17.945589 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.946078 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.946702 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.946944 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.946992 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.948887 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:17.948888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:17.948888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:17.948895 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:17.950083 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:17.950371 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:17.950372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:17.950372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:17.950425 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:17.950644 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:17.950645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:17.950645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:17.950734 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:17.950804 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:17.950881 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 20:13:17.950935 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:13:17.952678 4143 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:13:17.954304 4143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:13:17.960516 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.956621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:34733 (local address 127.0.0.1:34404)
0807 20:13:17.957018 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:17.957026 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:17.957069 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:17.957332 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:17.957335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:17.957338 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:17.957410 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.957412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.958277 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.958279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.959416 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.959419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.960201 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.960203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.960294 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.960309 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 20:13:17.960376 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 20:13:17.960438 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":99,"threads_started":1}
I20260807 20:13:17.960536 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.956890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34404 (local address 127.23.200.195:34733)
0807 20:13:17.957071 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:17.957076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:17.957116 (+ 40us) server_negotiation.cc:407] Connection header received
0807 20:13:17.957157 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:17.957159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:17.957175 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:17.957248 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:17.957489 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.958200 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.959509 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.960129 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.960207 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.960270 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:17.960397 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 20:13:17.960437 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":114,"threads_started":1}
I20260807 20:13:17.960800 4983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.956616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:42787 (local address 127.0.0.1:43150)
0807 20:13:17.956811 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:17.956849 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:17.956914 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:17.957576 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:17.957576 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:17.957578 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:17.957611 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.957612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.958827 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.958828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.959656 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.959659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.960571 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.960572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.960635 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.960639 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:13:17.960668 (+ 29us) client_negotiation.cc:240] Negotiation successful
0807 20:13:17.960706 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":82,"threads_started":1}
I20260807 20:13:17.960816 4981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.956616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:33353 (local address 127.0.0.1:53226)
0807 20:13:17.956810 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:17.956847 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:17.956913 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:17.957334 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:17.957335 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:17.957341 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:17.957411 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.957414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.958714 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.958716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.959768 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:17.959770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.960602 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.960603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:17.960652 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.960655 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:13:17.960681 (+ 26us) client_negotiation.cc:240] Negotiation successful
0807 20:13:17.960711 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":2,"thread_start_us":63,"threads_started":1}
I20260807 20:13:17.961021 4984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.956902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53226 (local address 127.23.200.194:33353)
0807 20:13:17.957190 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:17.957195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:17.957205 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:13:17.957238 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:17.957240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:17.957253 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:17.957323 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:17.957500 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.958616 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.959847 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.960519 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.960769 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.960847 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:17.960875 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 20:13:17.960916 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":79,"threads_started":1}
I20260807 20:13:17.961282 4828 tablet_service.cc:1467] Processing CreateTablet for tablet 96febbc0501f4c30b858cf4e3c1e093a (DEFAULT_TABLE table=test-table [id=60039bbaccc441b5a711652ead174788]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:17.961664 4828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96febbc0501f4c30b858cf4e3c1e093a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:17.961709 4609 tablet_service.cc:1467] Processing CreateTablet for tablet 96febbc0501f4c30b858cf4e3c1e093a (DEFAULT_TABLE table=test-table [id=60039bbaccc441b5a711652ead174788]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:17.961951 4986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:17.957132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43150 (local address 127.23.200.193:42787)
0807 20:13:17.957279 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:17.957284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:17.957294 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:13:17.957460 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:17.957463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:17.957481 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:17.957676 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:17.957730 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.958752 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.959730 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:17.960464 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:17.961584 (+ 1120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:17.961646 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:17.961672 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 20:13:17.961851 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260807 20:13:17.962093 4609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96febbc0501f4c30b858cf4e3c1e093a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:17.962555 4368 tablet_service.cc:1467] Processing CreateTablet for tablet 96febbc0501f4c30b858cf4e3c1e093a (DEFAULT_TABLE table=test-table [id=60039bbaccc441b5a711652ead174788]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:17.962872 4368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96febbc0501f4c30b858cf4e3c1e093a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:17.964740 4987 tablet_bootstrap.cc:492] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241: Bootstrap starting.
I20260807 20:13:17.965408 4987 tablet_bootstrap.cc:654] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:17.965698 4987 log.cc:824] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:17.966389 4987 tablet_bootstrap.cc:492] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241: No bootstrap required, opened a new log
I20260807 20:13:17.966457 4987 ts_tablet_manager.cc:1397] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:13:17.966518 4995 tablet_bootstrap.cc:492] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae: Bootstrap starting.
I20260807 20:13:17.966893 4989 tablet_bootstrap.cc:492] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d: Bootstrap starting.
I20260807 20:13:17.967162 4995 tablet_bootstrap.cc:654] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:17.967486 4995 log.cc:824] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:17.967512 4989 tablet_bootstrap.cc:654] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:17.967816 4989 log.cc:824] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:17.968161 4987 raft_consensus.cc:348] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } }
I20260807 20:13:17.968273 4995 tablet_bootstrap.cc:492] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae: No bootstrap required, opened a new log
I20260807 20:13:17.968318 4987 raft_consensus.cc:374] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:17.968351 4995 ts_tablet_manager.cc:1397] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:13:17.968351 4987 raft_consensus.cc:729] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61087001c28c4214a2a97c681f874241, State: Initialized, Role: FOLLOWER
I20260807 20:13:17.968492 4987 consensus_queue.cc:260] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } }
I20260807 20:13:17.968730 4987 ts_tablet_manager.cc:1428] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 20:13:17.968776 4944 heartbeater.cc:503] Master 127.23.200.254:42397 was elected leader, sending a full tablet report...
I20260807 20:13:17.969694 4989 tablet_bootstrap.cc:492] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d: No bootstrap required, opened a new log
I20260807 20:13:17.969764 4989 ts_tablet_manager.cc:1397] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 20:13:17.970290 4995 raft_consensus.cc:348] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } }
I20260807 20:13:17.970449 4995 raft_consensus.cc:374] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:17.970489 4995 raft_consensus.cc:729] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23c49522daf54d36a1f06d5ccd2a14ae, State: Initialized, Role: FOLLOWER
I20260807 20:13:17.970598 4995 consensus_queue.cc:260] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } }
I20260807 20:13:17.970880 4995 ts_tablet_manager.cc:1428] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 20:13:17.970888 4482 heartbeater.cc:503] Master 127.23.200.254:42397 was elected leader, sending a full tablet report...
I20260807 20:13:17.971755 4989 raft_consensus.cc:348] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } }
I20260807 20:13:17.971884 4989 raft_consensus.cc:374] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:17.971905 4989 raft_consensus.cc:729] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e8c651e0244c8ebd2828fd27019c4d, State: Initialized, Role: FOLLOWER
I20260807 20:13:17.971977 4989 consensus_queue.cc:260] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } }
I20260807 20:13:17.972152 4989 ts_tablet_manager.cc:1428] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 20:13:17.972205 4716 heartbeater.cc:503] Master 127.23.200.254:42397 was elected leader, sending a full tablet report...
W20260807 20:13:18.015323 4718 tablet.cc:2369] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:13:18.105296 5008 raft_consensus.cc:482] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:13:18.105441 5008 raft_consensus.cc:504] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } }
I20260807 20:13:18.105764 5008 leader_election.cc:288] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39e8c651e0244c8ebd2828fd27019c4d (127.23.200.194:33353), 23c49522daf54d36a1f06d5ccd2a14ae (127.23.200.193:42787)
W20260807 20:13:18.108093 4485 tablet.cc:2369] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:13:18.108620 5016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.105971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:33353 (local address 127.23.200.195:49391)
0807 20:13:18.106275 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:18.106284 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:18.106329 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:18.106435 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:18.106436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:18.106448 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:18.106520 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.106522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.107119 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.107119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.107819 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.107820 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.108374 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.108376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.108468 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.108472 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:13:18.108518 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 20:13:18.108549 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":110,"thread_start_us":136,"threads_started":1}
I20260807 20:13:18.108645 4984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.106115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:49391 (local address 127.23.200.194:33353)
0807 20:13:18.106174 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:18.106176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:18.106338 (+ 162us) server_negotiation.cc:407] Connection header received
0807 20:13:18.106371 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:18.106372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:18.106382 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:18.106423 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:18.106579 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.107051 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.107902 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.108313 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.108367 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.108375 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:18.108536 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 20:13:18.108572 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 20:13:18.108722 4986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.106046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:59651 (local address 127.23.200.193:42787)
0807 20:13:18.106107 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:18.106109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:18.106119 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:13:18.106147 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:18.106148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:18.106158 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:18.106213 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:18.106386 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.106931 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.107872 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.108386 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.108468 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.108476 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:18.108626 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 20:13:18.108664 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:13:18.108729 4765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.105968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:42787 (local address 127.23.200.195:59651)
0807 20:13:18.106022 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:18.106034 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:18.106084 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:18.106233 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:18.106235 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:18.106237 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:18.106303 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.106305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.107012 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.107013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.107789 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.107792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.108478 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.108479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.108555 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.108559 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:13:18.108613 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 20:13:18.108651 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 20:13:18.108994 4650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96febbc0501f4c30b858cf4e3c1e093a" candidate_uuid: "61087001c28c4214a2a97c681f874241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39e8c651e0244c8ebd2828fd27019c4d" is_pre_election: true
I20260807 20:13:18.109051 4401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96febbc0501f4c30b858cf4e3c1e093a" candidate_uuid: "61087001c28c4214a2a97c681f874241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" is_pre_election: true
I20260807 20:13:18.109198 4650 raft_consensus.cc:2454] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61087001c28c4214a2a97c681f874241 in term 0.
I20260807 20:13:18.109198 4401 raft_consensus.cc:2454] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61087001c28c4214a2a97c681f874241 in term 0.
I20260807 20:13:18.109433 4755 leader_election.cc:302] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39e8c651e0244c8ebd2828fd27019c4d, 61087001c28c4214a2a97c681f874241; no voters:
I20260807 20:13:18.109604 5008 raft_consensus.cc:2781] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:13:18.109656 5008 raft_consensus.cc:482] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:13:18.109671 5008 raft_consensus.cc:3037] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:18.110628 5008 raft_consensus.cc:504] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } }
I20260807 20:13:18.110740 5008 leader_election.cc:288] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [CANDIDATE]: Term 1 election: Requested vote from peers 39e8c651e0244c8ebd2828fd27019c4d (127.23.200.194:33353), 23c49522daf54d36a1f06d5ccd2a14ae (127.23.200.193:42787)
I20260807 20:13:18.110967 4401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96febbc0501f4c30b858cf4e3c1e093a" candidate_uuid: "61087001c28c4214a2a97c681f874241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c49522daf54d36a1f06d5ccd2a14ae"
I20260807 20:13:18.110993 4650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96febbc0501f4c30b858cf4e3c1e093a" candidate_uuid: "61087001c28c4214a2a97c681f874241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39e8c651e0244c8ebd2828fd27019c4d"
I20260807 20:13:18.111037 4401 raft_consensus.cc:3037] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:18.111073 4650 raft_consensus.cc:3037] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:18.111726 4401 raft_consensus.cc:2454] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61087001c28c4214a2a97c681f874241 in term 1.
I20260807 20:13:18.111886 4754 leader_election.cc:302] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23c49522daf54d36a1f06d5ccd2a14ae, 61087001c28c4214a2a97c681f874241; no voters:
I20260807 20:13:18.111896 4650 raft_consensus.cc:2454] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61087001c28c4214a2a97c681f874241 in term 1.
I20260807 20:13:18.111972 5008 raft_consensus.cc:2781] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:18.112129 5008 raft_consensus.cc:686] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [term 1 LEADER]: Becoming Leader. State: Replica: 61087001c28c4214a2a97c681f874241, State: Running, Role: LEADER
I20260807 20:13:18.112221 5008 consensus_queue.cc:237] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } }
I20260807 20:13:18.113054 4143 catalog_manager.cc:5495] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 reported cstate change: term changed from 0 to 1, leader changed from <none> to 61087001c28c4214a2a97c681f874241 (127.23.200.195). New cstate: current_term: 1 leader_uuid: "61087001c28c4214a2a97c681f874241" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } health_report { overall_health: UNKNOWN } } }
I20260807 20:13:18.142334 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.140196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34412 (local address 127.23.200.195:34733)
0807 20:13:18.140247 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:18.140249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:18.140257 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:13:18.140316 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:18.140316 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:18.140333 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:18.140397 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:13:18.140549 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.141016 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.141732 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.141922 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.141971 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.142086 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:13:18.142187 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:13:18.142246 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:18.142258 (+ 12us) server_negotiation.cc:299] Negotiation successful
0807 20:13:18.142276 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 20:13:18.144181 4401 raft_consensus.cc:1261] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Refusing update from remote peer 61087001c28c4214a2a97c681f874241: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:18.144181 4650 raft_consensus.cc:1261] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d [term 1 FOLLOWER]: Refusing update from remote peer 61087001c28c4214a2a97c681f874241: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:18.144587 5018 consensus_queue.cc:1035] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:18.144721 5008 consensus_queue.cc:1035] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:18.147684 5028 mvcc.cc:205] Tried to move back new op lower bound from 7316003217995939840 to 7316003217868267520. Current Snapshot: MvccSnapshot[applied={T|T < 7316003217995939840}]
I20260807 20:13:18.147833 5024 mvcc.cc:205] Tried to move back new op lower bound from 7316003217995939840 to 7316003217868267520. Current Snapshot: MvccSnapshot[applied={T|T < 7316003217995939840 or (T in {7316003217995939840})}]
I20260807 20:13:18.147861 5023 mvcc.cc:205] Tried to move back new op lower bound from 7316003217995939840 to 7316003217868267520. Current Snapshot: MvccSnapshot[applied={T|T < 7316003217995939840}]
I20260807 20:13:18.149853 4145 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:18.149943 4145 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:18.150662 4145 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a: Sending DeleteTablet for 3 replicas of tablet 96febbc0501f4c30b858cf4e3c1e093a
I20260807 20:13:18.150969 4828 tablet_service.cc:1514] Processing DeleteTablet for tablet 96febbc0501f4c30b858cf4e3c1e093a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:18 UTC) from {username='oryx', principal='oryx/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:34404
I20260807 20:13:18.151042 4368 tablet_service.cc:1514] Processing DeleteTablet for tablet 96febbc0501f4c30b858cf4e3c1e093a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:18 UTC) from {username='oryx', principal='oryx/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:43150
I20260807 20:13:18.151273 4145 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 20:13:18.151358 4609 tablet_service.cc:1514] Processing DeleteTablet for tablet 96febbc0501f4c30b858cf4e3c1e093a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:18 UTC) from {username='oryx', principal='oryx/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:53226
W20260807 20:13:18.151419 4145 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:13:18.151563 5044 tablet_replica.cc:330] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d: stopping tablet replica
I20260807 20:13:18.151659 5044 raft_consensus.cc:2229] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:18.151773 5044 raft_consensus.cc:2258] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:18.151924 5043 tablet_replica.cc:330] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae: stopping tablet replica
I20260807 20:13:18.152024 5043 raft_consensus.cc:2229] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:18.152118 5043 raft_consensus.cc:2258] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:18.152439 5042 tablet_replica.cc:330] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241: stopping tablet replica
I20260807 20:13:18.152534 5042 raft_consensus.cc:2229] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:13:18.152630 5044 ts_tablet_manager.cc:1905] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:18.152657 5042 raft_consensus.cc:2258] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:18.152686 4828 tablet_service.cc:1467] Processing CreateTablet for tablet 4fa69dc454e748399594312f64ec8fd6 (DEFAULT_TABLE table=test-table [id=d57d712963ca496a87a4faabfa7be136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:18.152817 4828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fa69dc454e748399594312f64ec8fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:18.153102 5042 ts_tablet_manager.cc:1905] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:18.154177 4987 tablet_bootstrap.cc:492] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241: Bootstrap starting.
I20260807 20:13:18.154589 5044 ts_tablet_manager.cc:1918] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:13:18.154618 4987 tablet_bootstrap.cc:654] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:18.154656 5044 log.cc:1196] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/96febbc0501f4c30b858cf4e3c1e093a
I20260807 20:13:18.154637 5042 ts_tablet_manager.cc:1918] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:13:18.154703 5042 log.cc:1196] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/96febbc0501f4c30b858cf4e3c1e093a
I20260807 20:13:18.154902 5044 ts_tablet_manager.cc:1939] T 96febbc0501f4c30b858cf4e3c1e093a P 39e8c651e0244c8ebd2828fd27019c4d: Deleting consensus metadata
I20260807 20:13:18.154914 5042 ts_tablet_manager.cc:1939] T 96febbc0501f4c30b858cf4e3c1e093a P 61087001c28c4214a2a97c681f874241: Deleting consensus metadata
I20260807 20:13:18.155179 4368 tablet_service.cc:1467] Processing CreateTablet for tablet 4fa69dc454e748399594312f64ec8fd6 (DEFAULT_TABLE table=test-table [id=d57d712963ca496a87a4faabfa7be136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:18.155289 4368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fa69dc454e748399594312f64ec8fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:18.155292 4122 catalog_manager.cc:4841] TS 39e8c651e0244c8ebd2828fd27019c4d (127.23.200.194:33353): tablet 96febbc0501f4c30b858cf4e3c1e093a (table test-table [id=60039bbaccc441b5a711652ead174788]) successfully deleted
I20260807 20:13:18.155344 4609 tablet_service.cc:1467] Processing CreateTablet for tablet 4fa69dc454e748399594312f64ec8fd6 (DEFAULT_TABLE table=test-table [id=d57d712963ca496a87a4faabfa7be136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:18.155448 4609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fa69dc454e748399594312f64ec8fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:18.156167 4995 tablet_bootstrap.cc:492] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae: Bootstrap starting.
I20260807 20:13:18.156530 4995 tablet_bootstrap.cc:654] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:18.156602 4989 tablet_bootstrap.cc:492] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d: Bootstrap starting.
I20260807 20:13:18.156672 4123 catalog_manager.cc:4841] TS 61087001c28c4214a2a97c681f874241 (127.23.200.195:34733): tablet 96febbc0501f4c30b858cf4e3c1e093a (table test-table [id=60039bbaccc441b5a711652ead174788]) successfully deleted
I20260807 20:13:18.156981 4989 tablet_bootstrap.cc:654] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:18.157166 4995 tablet_bootstrap.cc:492] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae: No bootstrap required, opened a new log
I20260807 20:13:18.157212 4995 ts_tablet_manager.cc:1397] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:18.157347 4995 raft_consensus.cc:348] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.157423 4995 raft_consensus.cc:374] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:18.157460 4995 raft_consensus.cc:729] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23c49522daf54d36a1f06d5ccd2a14ae, State: Initialized, Role: FOLLOWER
I20260807 20:13:18.157517 4989 tablet_bootstrap.cc:492] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d: No bootstrap required, opened a new log
I20260807 20:13:18.157501 4995 consensus_queue.cc:260] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.157562 4989 ts_tablet_manager.cc:1397] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 20:13:18.157618 4995 ts_tablet_manager.cc:1428] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:18.157698 4989 raft_consensus.cc:348] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.157766 4989 raft_consensus.cc:374] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:18.157784 4989 raft_consensus.cc:729] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e8c651e0244c8ebd2828fd27019c4d, State: Initialized, Role: FOLLOWER
I20260807 20:13:18.157826 4989 consensus_queue.cc:260] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.157915 4989 ts_tablet_manager.cc:1428] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:18.158411 4987 tablet_bootstrap.cc:492] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241: No bootstrap required, opened a new log
I20260807 20:13:18.158458 4987 ts_tablet_manager.cc:1397] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 20:13:18.158609 4987 raft_consensus.cc:348] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.158679 4987 raft_consensus.cc:374] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:18.158699 4987 raft_consensus.cc:729] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61087001c28c4214a2a97c681f874241, State: Initialized, Role: FOLLOWER
I20260807 20:13:18.158737 4987 consensus_queue.cc:260] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.158815 4987 ts_tablet_manager.cc:1428] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:18.161473 5043 ts_tablet_manager.cc:1905] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:18.163105 5043 ts_tablet_manager.cc:1918] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:13:18.163166 5043 log.cc:1196] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/wal/wals/96febbc0501f4c30b858cf4e3c1e093a
I20260807 20:13:18.163383 5043 ts_tablet_manager.cc:1939] T 96febbc0501f4c30b858cf4e3c1e093a P 23c49522daf54d36a1f06d5ccd2a14ae: Deleting consensus metadata
I20260807 20:13:18.163745 4117 catalog_manager.cc:4841] TS 23c49522daf54d36a1f06d5ccd2a14ae (127.23.200.193:42787): tablet 96febbc0501f4c30b858cf4e3c1e093a (table test-table [id=60039bbaccc441b5a711652ead174788]) successfully deleted
W20260807 20:13:18.167893 4946 tablet.cc:2369] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:13:18.386493 5013 raft_consensus.cc:482] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:13:18.386593 5013 raft_consensus.cc:504] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.386965 5013 leader_election.cc:288] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23c49522daf54d36a1f06d5ccd2a14ae (127.23.200.193:42787), 61087001c28c4214a2a97c681f874241 (127.23.200.195:34733)
I20260807 20:13:18.390807 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.387207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:51833 (local address 127.23.200.195:34733)
0807 20:13:18.387268 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:18.387271 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:18.387347 (+ 76us) server_negotiation.cc:407] Connection header received
0807 20:13:18.387431 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:18.387432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:18.387444 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:18.387495 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:18.387648 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.388423 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.389919 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.390474 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.390544 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.390553 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:18.390696 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 20:13:18.390733 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 20:13:18.390807 4986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.387193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:36589 (local address 127.23.200.193:42787)
0807 20:13:18.387273 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:18.387275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:18.387338 (+ 63us) server_negotiation.cc:407] Connection header received
0807 20:13:18.387416 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:18.387417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:18.387427 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:18.387496 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:18.387648 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.388426 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.389925 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.390467 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.390544 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.390553 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:18.390696 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 20:13:18.390735 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 20:13:18.390842 5053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.387128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:42787 (local address 127.23.200.194:36589)
0807 20:13:18.387265 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:18.387278 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:18.387357 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:18.387502 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:18.387503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:18.387505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:18.387586 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.387587 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.388494 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.388494 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.389832 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.389836 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.390550 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.390551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.390629 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.390633 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:13:18.390696 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 20:13:18.390733 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":29,"thread_start_us":61,"threads_started":1}
I20260807 20:13:18.390851 5052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.387131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:34733 (local address 127.23.200.194:51833)
0807 20:13:18.387267 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:18.387278 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:18.387357 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:18.387502 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:18.387503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:18.387506 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:18.387583 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.387585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.388494 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.388494 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.389832 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.389836 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.390549 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.390550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.390629 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.390634 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:13:18.390687 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 20:13:18.390733 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":78,"thread_start_us":61,"threads_started":1}
I20260807 20:13:18.391155 4401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fa69dc454e748399594312f64ec8fd6" candidate_uuid: "39e8c651e0244c8ebd2828fd27019c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" is_pre_election: true
I20260807 20:13:18.391251 4401 raft_consensus.cc:2454] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39e8c651e0244c8ebd2828fd27019c4d in term 0.
I20260807 20:13:18.391250 4861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fa69dc454e748399594312f64ec8fd6" candidate_uuid: "39e8c651e0244c8ebd2828fd27019c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61087001c28c4214a2a97c681f874241" is_pre_election: true
I20260807 20:13:18.391426 4861 raft_consensus.cc:2454] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39e8c651e0244c8ebd2828fd27019c4d in term 0.
I20260807 20:13:18.391422 4519 leader_election.cc:302] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23c49522daf54d36a1f06d5ccd2a14ae, 39e8c651e0244c8ebd2828fd27019c4d; no voters:
I20260807 20:13:18.391572 5013 raft_consensus.cc:2781] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:13:18.391613 5013 raft_consensus.cc:482] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:13:18.391631 5013 raft_consensus.cc:3037] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:18.392292 5013 raft_consensus.cc:504] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.392407 5013 leader_election.cc:288] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [CANDIDATE]: Term 1 election: Requested vote from peers 23c49522daf54d36a1f06d5ccd2a14ae (127.23.200.193:42787), 61087001c28c4214a2a97c681f874241 (127.23.200.195:34733)
I20260807 20:13:18.392552 4401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fa69dc454e748399594312f64ec8fd6" candidate_uuid: "39e8c651e0244c8ebd2828fd27019c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c49522daf54d36a1f06d5ccd2a14ae"
I20260807 20:13:18.392570 4861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fa69dc454e748399594312f64ec8fd6" candidate_uuid: "39e8c651e0244c8ebd2828fd27019c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61087001c28c4214a2a97c681f874241"
I20260807 20:13:18.392623 4401 raft_consensus.cc:3037] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:18.392627 4861 raft_consensus.cc:3037] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:18.393277 4401 raft_consensus.cc:2454] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39e8c651e0244c8ebd2828fd27019c4d in term 1.
I20260807 20:13:18.393353 4861 raft_consensus.cc:2454] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39e8c651e0244c8ebd2828fd27019c4d in term 1.
I20260807 20:13:18.393424 4519 leader_election.cc:302] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23c49522daf54d36a1f06d5ccd2a14ae, 39e8c651e0244c8ebd2828fd27019c4d; no voters:
I20260807 20:13:18.393533 5013 raft_consensus.cc:2781] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:18.393646 5013 raft_consensus.cc:686] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [term 1 LEADER]: Becoming Leader. State: Replica: 39e8c651e0244c8ebd2828fd27019c4d, State: Running, Role: LEADER
I20260807 20:13:18.393707 5013 consensus_queue.cc:237] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.394321 4145 catalog_manager.cc:5495] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 39e8c651e0244c8ebd2828fd27019c4d (127.23.200.194). New cstate: current_term: 1 leader_uuid: "39e8c651e0244c8ebd2828fd27019c4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } health_report { overall_health: UNKNOWN } } }
Aug 07 20:13:18 dist-test-slave-2j7r krb5kdc[4063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133597, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.254@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM
I20260807 20:13:18.464133 5068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.457831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58082 (local address 127.23.200.254:42397)
0807 20:13:18.457993 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:18.457995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:18.458005 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:13:18.458054 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:18.458054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:18.458064 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:18.458126 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:18.458571 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.459067 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.459706 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.459895 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.459941 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.461904 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:18.461906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:18.461906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:18.461912 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:18.463236 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:18.463563 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:18.463564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:18.463564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:18.463615 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:18.463755 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:18.463755 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:18.463755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:18.463848 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:18.463890 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:18.463995 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 20:13:18.464060 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":87,"threads_started":1}
I20260807 20:13:18.464202 5067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.457765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42397 (local address 127.0.0.1:58082)
0807 20:13:18.457920 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:18.457933 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:18.458001 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:18.458136 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:18.458137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:18.458464 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:18.458513 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.458515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.459120 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.459120 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.459652 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.459654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.459948 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.459949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.459989 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.460422 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:18.460426 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:18.461817 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:18.463298 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:18.463304 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:18.463306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:18.463509 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:18.463661 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:18.463661 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:18.463662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:18.463714 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:18.463887 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:18.463892 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:18.463930 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 20:13:18.463978 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 20:13:18.464110 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":34,"thread_start_us":72,"threads_started":1}
I20260807 20:13:18.465978 4145 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:58082:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 20:13:18.466166 4145 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:13:18.467494 4609 tablet_service.cc:1467] Processing CreateTablet for tablet 197998ad02d546bb8a46e3d0aac428a8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=02fd21f0e40a44939efbf2040f33fde7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:13:18.467499 4828 tablet_service.cc:1467] Processing CreateTablet for tablet 197998ad02d546bb8a46e3d0aac428a8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=02fd21f0e40a44939efbf2040f33fde7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:13:18.467514 4368 tablet_service.cc:1467] Processing CreateTablet for tablet 197998ad02d546bb8a46e3d0aac428a8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=02fd21f0e40a44939efbf2040f33fde7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:13:18.467664 4368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197998ad02d546bb8a46e3d0aac428a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:18.467664 4828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197998ad02d546bb8a46e3d0aac428a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:18.467662 4609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197998ad02d546bb8a46e3d0aac428a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:18.469067 4995 tablet_bootstrap.cc:492] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae: Bootstrap starting.
I20260807 20:13:18.469198 4987 tablet_bootstrap.cc:492] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241: Bootstrap starting.
I20260807 20:13:18.469218 4989 tablet_bootstrap.cc:492] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d: Bootstrap starting.
I20260807 20:13:18.469519 4987 tablet_bootstrap.cc:654] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:18.469576 4995 tablet_bootstrap.cc:654] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:18.469607 4989 tablet_bootstrap.cc:654] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:18.470073 4987 tablet_bootstrap.cc:492] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241: No bootstrap required, opened a new log
I20260807 20:13:18.470126 4987 ts_tablet_manager.cc:1397] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:18.470216 4989 tablet_bootstrap.cc:492] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d: No bootstrap required, opened a new log
I20260807 20:13:18.470255 4989 ts_tablet_manager.cc:1397] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 20:13:18.470233 4987 raft_consensus.cc:348] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.470284 4987 raft_consensus.cc:374] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:18.470296 4987 raft_consensus.cc:729] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61087001c28c4214a2a97c681f874241, State: Initialized, Role: FOLLOWER
I20260807 20:13:18.470330 4987 consensus_queue.cc:260] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.470362 4989 raft_consensus.cc:348] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.470404 4989 raft_consensus.cc:374] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:18.470417 4989 raft_consensus.cc:729] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e8c651e0244c8ebd2828fd27019c4d, State: Initialized, Role: FOLLOWER
I20260807 20:13:18.470415 4987 ts_tablet_manager.cc:1428] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:18.470443 5018 tablet_replica.cc:439] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } }
I20260807 20:13:18.470470 4989 consensus_queue.cc:260] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.470518 5018 tablet_replica.cc:442] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:13:18.470556 4989 ts_tablet_manager.cc:1428] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:18.470790 4995 tablet_bootstrap.cc:492] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae: No bootstrap required, opened a new log
I20260807 20:13:18.470851 4995 ts_tablet_manager.cc:1397] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 20:13:18.471002 4995 raft_consensus.cc:348] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.471064 4995 raft_consensus.cc:374] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:18.471076 4995 raft_consensus.cc:729] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23c49522daf54d36a1f06d5ccd2a14ae, State: Initialized, Role: FOLLOWER
I20260807 20:13:18.471107 4995 consensus_queue.cc:260] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.471165 4995 ts_tablet_manager.cc:1428] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:18.471220 5009 tablet_replica.cc:439] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } }
I20260807 20:13:18.471313 5009 tablet_replica.cc:442] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:13:18.471462 5013 tablet_replica.cc:439] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } }
I20260807 20:13:18.471545 5013 tablet_replica.cc:442] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:13:18.488904 5009 raft_consensus.cc:482] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:13:18.489008 5009 raft_consensus.cc:504] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.489434 5009 leader_election.cc:288] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39e8c651e0244c8ebd2828fd27019c4d (127.23.200.194:33353), 61087001c28c4214a2a97c681f874241 (127.23.200.195:34733)
I20260807 20:13:18.492548 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.489964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:39863 (local address 127.23.200.195:34733)
0807 20:13:18.490025 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:18.490028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:18.490122 (+ 94us) server_negotiation.cc:407] Connection header received
0807 20:13:18.490195 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:18.490197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:18.490207 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:18.490257 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:18.490435 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.490885 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.491876 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.492236 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.492299 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.492308 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:18.492447 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 20:13:18.492482 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 20:13:18.492568 5076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.489890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:34733 (local address 127.23.200.193:39863)
0807 20:13:18.490061 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:18.490070 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:18.490124 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:18.490268 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:18.490269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:18.490283 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:18.490354 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.490356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.490957 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.490958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.491785 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.491788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.492309 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.492310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.492383 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.492387 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:13:18.492435 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 20:13:18.492473 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260807 20:13:18.492844 4861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197998ad02d546bb8a46e3d0aac428a8" candidate_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61087001c28c4214a2a97c681f874241" is_pre_election: true
I20260807 20:13:18.492908 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.489613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:33353 (local address 127.23.200.193:50959)
0807 20:13:18.490060 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:18.490070 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:18.490135 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:18.490262 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:18.490263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:18.490265 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:18.490342 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.490344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.490967 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.490968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.491788 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.491791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.492665 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.492666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.492745 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.492748 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:13:18.492798 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 20:13:18.492829 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":127,"threads_started":1}
I20260807 20:13:18.492924 4984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.489690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:50959 (local address 127.23.200.194:33353)
0807 20:13:18.489751 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:18.489753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:18.490135 (+ 382us) server_negotiation.cc:407] Connection header received
0807 20:13:18.490198 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:18.490199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:18.490208 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:18.490249 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:18.490401 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.490885 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.491874 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.492236 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.492299 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.492306 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:18.492820 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 20:13:18.492865 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:13:18.492954 4861 raft_consensus.cc:2454] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23c49522daf54d36a1f06d5ccd2a14ae in term 0.
I20260807 20:13:18.493173 4294 leader_election.cc:302] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23c49522daf54d36a1f06d5ccd2a14ae, 61087001c28c4214a2a97c681f874241; no voters:
I20260807 20:13:18.493257 4650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197998ad02d546bb8a46e3d0aac428a8" candidate_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39e8c651e0244c8ebd2828fd27019c4d" is_pre_election: true
I20260807 20:13:18.493311 5009 raft_consensus.cc:2781] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:13:18.493359 5009 raft_consensus.cc:482] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:13:18.493352 4650 raft_consensus.cc:2454] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23c49522daf54d36a1f06d5ccd2a14ae in term 0.
I20260807 20:13:18.493386 5009 raft_consensus.cc:3037] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:18.494069 5009 raft_consensus.cc:504] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.494182 5009 leader_election.cc:288] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [CANDIDATE]: Term 1 election: Requested vote from peers 39e8c651e0244c8ebd2828fd27019c4d (127.23.200.194:33353), 61087001c28c4214a2a97c681f874241 (127.23.200.195:34733)
I20260807 20:13:18.494316 4650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197998ad02d546bb8a46e3d0aac428a8" candidate_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39e8c651e0244c8ebd2828fd27019c4d"
I20260807 20:13:18.494400 4650 raft_consensus.cc:3037] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:18.494364 4861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197998ad02d546bb8a46e3d0aac428a8" candidate_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61087001c28c4214a2a97c681f874241"
I20260807 20:13:18.494464 4861 raft_consensus.cc:3037] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:18.495026 4650 raft_consensus.cc:2454] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23c49522daf54d36a1f06d5ccd2a14ae in term 1.
I20260807 20:13:18.495042 4861 raft_consensus.cc:2454] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23c49522daf54d36a1f06d5ccd2a14ae in term 1.
I20260807 20:13:18.495198 4293 leader_election.cc:302] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23c49522daf54d36a1f06d5ccd2a14ae, 39e8c651e0244c8ebd2828fd27019c4d; no voters:
I20260807 20:13:18.495301 5009 raft_consensus.cc:2781] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:18.495460 5009 raft_consensus.cc:686] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 LEADER]: Becoming Leader. State: Replica: 23c49522daf54d36a1f06d5ccd2a14ae, State: Running, Role: LEADER
I20260807 20:13:18.495537 5009 consensus_queue.cc:237] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.495870 5079 tablet_replica.cc:439] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae: TxnStatusTablet state changed. Reason: New leader 23c49522daf54d36a1f06d5ccd2a14ae. Latest consensus state: current_term: 1 leader_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } }
I20260807 20:13:18.495965 5079 tablet_replica.cc:442] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:13:18.496202 5087 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 20:13:18.496181 4145 catalog_manager.cc:5495] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 23c49522daf54d36a1f06d5ccd2a14ae (127.23.200.193). New cstate: current_term: 1 leader_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } health_report { overall_health: UNKNOWN } } }
I20260807 20:13:18.496315 5087 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 20:13:18.508702 5067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.505916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:42787 (local address 127.0.0.1:43154)
0807 20:13:18.506001 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:18.506013 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:18.506056 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:18.506370 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:18.506371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:18.506372 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:13:18.506447 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.506450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.507292 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.507292 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.507860 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.507862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.508283 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.508284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.508320 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.508334 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:13:18.508581 (+ 247us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:13:18.508585 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:13:18.508610 (+ 25us) client_negotiation.cc:240] Negotiation successful
0807 20:13:18.508646 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 20:13:18.508703 4986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.506143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43154 (local address 127.23.200.193:42787)
0807 20:13:18.506204 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:18.506206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:18.506215 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:18.506262 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:18.506263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:18.506283 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:18.506349 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:13:18.506503 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.507227 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.507925 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.508220 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.508279 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.508411 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:13:18.508526 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:13:18.508593 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:18.508613 (+ 20us) server_negotiation.cc:299] Negotiation successful
0807 20:13:18.508630 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 20:13:18.509783 4650 raft_consensus.cc:1261] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d [term 1 FOLLOWER]: Refusing update from remote peer 23c49522daf54d36a1f06d5ccd2a14ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:18.509931 4861 raft_consensus.cc:1261] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Refusing update from remote peer 23c49522daf54d36a1f06d5ccd2a14ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:18.510215 5079 consensus_queue.cc:1035] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:18.510342 5009 consensus_queue.cc:1035] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:18.510929 5013 tablet_replica.cc:439] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d: TxnStatusTablet state changed. Reason: New leader 23c49522daf54d36a1f06d5ccd2a14ae. Latest consensus state: current_term: 1 leader_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } }
I20260807 20:13:18.511018 5013 tablet_replica.cc:442] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:13:18.510967 5018 tablet_replica.cc:439] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241: TxnStatusTablet state changed. Reason: New leader 23c49522daf54d36a1f06d5ccd2a14ae. Latest consensus state: current_term: 1 leader_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } }
I20260807 20:13:18.511047 5018 tablet_replica.cc:442] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:13:18.511885 5009 tablet_replica.cc:439] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } }
I20260807 20:13:18.511965 5009 tablet_replica.cc:442] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:13:18.512007 5009 tablet_replica.cc:439] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } }
I20260807 20:13:18.512045 5009 tablet_replica.cc:442] T 197998ad02d546bb8a46e3d0aac428a8 P 23c49522daf54d36a1f06d5ccd2a14ae: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:13:18.512118 5013 tablet_replica.cc:439] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } }
I20260807 20:13:18.512181 5013 tablet_replica.cc:442] T 197998ad02d546bb8a46e3d0aac428a8 P 39e8c651e0244c8ebd2828fd27019c4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:13:18.512998 5018 tablet_replica.cc:439] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } } }
I20260807 20:13:18.513164 5018 tablet_replica.cc:442] T 197998ad02d546bb8a46e3d0aac428a8 P 61087001c28c4214a2a97c681f874241: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:13:18.516260 4984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.513602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53238 (local address 127.23.200.194:33353)
0807 20:13:18.513636 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:18.513638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:18.513647 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:18.513690 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:18.513691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:18.513711 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:18.513790 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:13:18.513874 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.514629 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.515374 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.515679 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.515750 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.515948 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:13:18.516078 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:13:18.516151 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:18.516171 (+ 20us) server_negotiation.cc:299] Negotiation successful
0807 20:13:18.516190 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260807 20:13:18.521559 5103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.518674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:42787 (local address 127.23.200.194:47235)
0807 20:13:18.518884 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:18.518895 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:18.518953 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:18.519073 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:18.519074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:18.519076 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:13:18.519124 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.519125 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.519997 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.519998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.520648 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.520651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.521169 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.521170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.521235 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.521251 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:13:18.521428 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:13:18.521430 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:13:18.521461 (+ 31us) client_negotiation.cc:240] Negotiation successful
0807 20:13:18.521492 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":89,"threads_started":1}
I20260807 20:13:18.521560 4986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.518743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:47235 (local address 127.23.200.193:42787)
0807 20:13:18.518796 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:18.518798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:18.518955 (+ 157us) server_negotiation.cc:407] Connection header received
0807 20:13:18.518997 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:18.518998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:18.519008 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:18.519072 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:13:18.519180 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.519895 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.520724 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.521022 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.521145 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.521330 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:13:18.521363 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:13:18.521428 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:18.521469 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 20:13:18.521489 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 20:13:18.523895 4401 raft_consensus.cc:1261] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Refusing update from remote peer 39e8c651e0244c8ebd2828fd27019c4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:18.523948 4861 raft_consensus.cc:1261] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Refusing update from remote peer 39e8c651e0244c8ebd2828fd27019c4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:18.524137 5013 consensus_queue.cc:1035] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:18.524469 5055 consensus_queue.cc:1035] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:18.533524 4984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.530707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:42971 (local address 127.23.200.194:33353)
0807 20:13:18.530742 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:18.530745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:18.530870 (+ 125us) server_negotiation.cc:407] Connection header received
0807 20:13:18.530905 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:18.530906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:18.530916 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:18.530980 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:13:18.531089 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.531823 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.532782 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.533111 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.533177 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.533317 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:13:18.533350 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:13:18.533395 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:18.533432 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 20:13:18.533451 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:13:18.533535 5113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.530661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:33353 (local address 127.23.200.193:42971)
0807 20:13:18.530805 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:18.530815 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:18.530867 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:18.530981 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:18.530982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:18.530983 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:13:18.531041 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.531042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.531905 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.531907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.532715 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:18.532718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.533188 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.533189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:18.533236 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.533250 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:13:18.533398 (+ 148us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:13:18.533399 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 20:13:18.533429 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 20:13:18.533466 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260807 20:13:18.542511 4143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:18.542610 4143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:18.543190 4143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a: Sending DeleteTablet for 3 replicas of tablet 4fa69dc454e748399594312f64ec8fd6
I20260807 20:13:18.543452 4828 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fa69dc454e748399594312f64ec8fd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:18 UTC) from {username='oryx', principal='oryx/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:34404
I20260807 20:13:18.543462 4609 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fa69dc454e748399594312f64ec8fd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:18 UTC) from {username='oryx', principal='oryx/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:53226
I20260807 20:13:18.543496 4368 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fa69dc454e748399594312f64ec8fd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:18 UTC) from {username='oryx', principal='oryx/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:43150
I20260807 20:13:18.543553 5042 tablet_replica.cc:330] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241: stopping tablet replica
I20260807 20:13:18.543570 5044 tablet_replica.cc:330] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d: stopping tablet replica
I20260807 20:13:18.543574 5043 tablet_replica.cc:330] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae: stopping tablet replica
I20260807 20:13:18.543603 5042 raft_consensus.cc:2229] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:18.543613 5044 raft_consensus.cc:2229] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:13:18.543617 5043 raft_consensus.cc:2229] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:18.543641 5042 raft_consensus.cc:2258] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:18.543645 5043 raft_consensus.cc:2258] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:18.543696 5044 raft_consensus.cc:2258] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:18.543949 5042 ts_tablet_manager.cc:1905] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:18.544013 5044 ts_tablet_manager.cc:1905] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:18.545989 5042 ts_tablet_manager.cc:1918] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:13:18.546051 5042 log.cc:1196] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/4fa69dc454e748399594312f64ec8fd6
I20260807 20:13:18.546272 5042 ts_tablet_manager.cc:1939] T 4fa69dc454e748399594312f64ec8fd6 P 61087001c28c4214a2a97c681f874241: Deleting consensus metadata
I20260807 20:13:18.546607 4123 catalog_manager.cc:4841] TS 61087001c28c4214a2a97c681f874241 (127.23.200.195:34733): tablet 4fa69dc454e748399594312f64ec8fd6 (table test-table [id=d57d712963ca496a87a4faabfa7be136]) successfully deleted
I20260807 20:13:18.546762 5044 ts_tablet_manager.cc:1918] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:13:18.546813 5044 log.cc:1196] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/4fa69dc454e748399594312f64ec8fd6
I20260807 20:13:18.547019 5044 ts_tablet_manager.cc:1939] T 4fa69dc454e748399594312f64ec8fd6 P 39e8c651e0244c8ebd2828fd27019c4d: Deleting consensus metadata
I20260807 20:13:18.547339 4122 catalog_manager.cc:4841] TS 39e8c651e0244c8ebd2828fd27019c4d (127.23.200.194:33353): tablet 4fa69dc454e748399594312f64ec8fd6 (table test-table [id=d57d712963ca496a87a4faabfa7be136]) successfully deleted
I20260807 20:13:18.547926 5043 ts_tablet_manager.cc:1905] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:18.549250 5043 ts_tablet_manager.cc:1918] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:13:18.549319 5043 log.cc:1196] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/wal/wals/4fa69dc454e748399594312f64ec8fd6
I20260807 20:13:18.549515 5043 ts_tablet_manager.cc:1939] T 4fa69dc454e748399594312f64ec8fd6 P 23c49522daf54d36a1f06d5ccd2a14ae: Deleting consensus metadata
I20260807 20:13:18.549839 4117 catalog_manager.cc:4841] TS 23c49522daf54d36a1f06d5ccd2a14ae (127.23.200.193:42787): tablet 4fa69dc454e748399594312f64ec8fd6 (table test-table [id=d57d712963ca496a87a4faabfa7be136]) successfully deleted
Aug 07 20:13:18 dist-test-slave-2j7r krb5kdc[4063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 20:13:18 dist-test-slave-2j7r krb5kdc[4063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:18 dist-test-slave-2j7r krb5kdc[4063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 20:13:18.584215 5068 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:18.577362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58094 (local address 127.23.200.254:42397)
0807 20:13:18.577429 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:18.577431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:18.577554 (+ 123us) server_negotiation.cc:407] Connection header received
0807 20:13:18.577616 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:18.577617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:18.577633 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:18.577705 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:18.578426 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.579183 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.580283 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.580612 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.580690 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.584145 (+ 3455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58094: BlockingRecv error: recv got EOF from 127.0.0.1:58094 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 07 20:13:18 dist-test-slave-2j7r krb5kdc[4063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM
I20260807 20:13:18.597853 5068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.585379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58108 (local address 127.23.200.254:42397)
0807 20:13:18.585421 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:18.585423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:18.589252 (+ 3829us) server_negotiation.cc:407] Connection header received
0807 20:13:18.589310 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:18.589311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:18.589324 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:18.589392 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:18.589972 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.590730 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.591754 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.592052 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.592131 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.594831 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:18.594833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:18.594834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:18.594841 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:18.596480 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:18.596885 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:18.596886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:18.596886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:18.596959 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:18.597185 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:18.597186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:18.597187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:18.597328 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:18.597433 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:18.597664 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 20:13:18.597753 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:13:18.599550 4143 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:13:18.599728 4143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:13:18.601161 4368 tablet_service.cc:1467] Processing CreateTablet for tablet 81e74d3bc41b4040bbaeb2b07d2ae50d (DEFAULT_TABLE table=test-table [id=6527bd5a627741c39a83eaf1958eb970]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:18.601310 4368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81e74d3bc41b4040bbaeb2b07d2ae50d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:18.601276 4828 tablet_service.cc:1467] Processing CreateTablet for tablet 81e74d3bc41b4040bbaeb2b07d2ae50d (DEFAULT_TABLE table=test-table [id=6527bd5a627741c39a83eaf1958eb970]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:18.601408 4828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81e74d3bc41b4040bbaeb2b07d2ae50d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:18.602669 4987 tablet_bootstrap.cc:492] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241: Bootstrap starting.
I20260807 20:13:18.602739 4609 tablet_service.cc:1467] Processing CreateTablet for tablet 81e74d3bc41b4040bbaeb2b07d2ae50d (DEFAULT_TABLE table=test-table [id=6527bd5a627741c39a83eaf1958eb970]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:18.602852 4609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81e74d3bc41b4040bbaeb2b07d2ae50d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:18.602999 4987 tablet_bootstrap.cc:654] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:18.603543 4987 tablet_bootstrap.cc:492] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241: No bootstrap required, opened a new log
I20260807 20:13:18.603593 4987 ts_tablet_manager.cc:1397] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:18.603727 4987 raft_consensus.cc:348] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.603790 4987 raft_consensus.cc:374] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:18.603809 4987 raft_consensus.cc:729] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61087001c28c4214a2a97c681f874241, State: Initialized, Role: FOLLOWER
I20260807 20:13:18.603848 4987 consensus_queue.cc:260] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.603950 4987 ts_tablet_manager.cc:1428] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:18.604183 4995 tablet_bootstrap.cc:492] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae: Bootstrap starting.
I20260807 20:13:18.604605 4995 tablet_bootstrap.cc:654] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:18.605050 4989 tablet_bootstrap.cc:492] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d: Bootstrap starting.
I20260807 20:13:18.605561 4989 tablet_bootstrap.cc:654] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:18.606678 4995 tablet_bootstrap.cc:492] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae: No bootstrap required, opened a new log
I20260807 20:13:18.606734 4995 ts_tablet_manager.cc:1397] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260807 20:13:18.606884 4995 raft_consensus.cc:348] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.606979 4995 raft_consensus.cc:374] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:18.607000 4995 raft_consensus.cc:729] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23c49522daf54d36a1f06d5ccd2a14ae, State: Initialized, Role: FOLLOWER
I20260807 20:13:18.607039 4995 consensus_queue.cc:260] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.607141 4995 ts_tablet_manager.cc:1428] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 20:13:18.608003 4989 tablet_bootstrap.cc:492] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d: No bootstrap required, opened a new log
I20260807 20:13:18.608047 4989 ts_tablet_manager.cc:1397] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260807 20:13:18.608176 4989 raft_consensus.cc:348] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.608232 4989 raft_consensus.cc:374] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:18.608248 4989 raft_consensus.cc:729] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e8c651e0244c8ebd2828fd27019c4d, State: Initialized, Role: FOLLOWER
I20260807 20:13:18.608285 4989 consensus_queue.cc:260] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.608359 4989 ts_tablet_manager.cc:1428] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:18.710897 5109 raft_consensus.cc:482] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:13:18.711015 5109 raft_consensus.cc:504] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.711200 5109 leader_election.cc:288] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23c49522daf54d36a1f06d5ccd2a14ae (127.23.200.193:42787), 61087001c28c4214a2a97c681f874241 (127.23.200.195:34733)
I20260807 20:13:18.711459 4401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81e74d3bc41b4040bbaeb2b07d2ae50d" candidate_uuid: "39e8c651e0244c8ebd2828fd27019c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" is_pre_election: true
I20260807 20:13:18.711479 4861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81e74d3bc41b4040bbaeb2b07d2ae50d" candidate_uuid: "39e8c651e0244c8ebd2828fd27019c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61087001c28c4214a2a97c681f874241" is_pre_election: true
I20260807 20:13:18.711580 4861 raft_consensus.cc:2454] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39e8c651e0244c8ebd2828fd27019c4d in term 0.
I20260807 20:13:18.711592 4401 raft_consensus.cc:2454] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39e8c651e0244c8ebd2828fd27019c4d in term 0.
I20260807 20:13:18.711759 4519 leader_election.cc:302] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23c49522daf54d36a1f06d5ccd2a14ae, 39e8c651e0244c8ebd2828fd27019c4d; no voters:
I20260807 20:13:18.711854 5109 raft_consensus.cc:2781] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:13:18.711894 5109 raft_consensus.cc:482] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:13:18.711912 5109 raft_consensus.cc:3037] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:18.712620 5109 raft_consensus.cc:504] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.712736 5109 leader_election.cc:288] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [CANDIDATE]: Term 1 election: Requested vote from peers 23c49522daf54d36a1f06d5ccd2a14ae (127.23.200.193:42787), 61087001c28c4214a2a97c681f874241 (127.23.200.195:34733)
I20260807 20:13:18.712888 4861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81e74d3bc41b4040bbaeb2b07d2ae50d" candidate_uuid: "39e8c651e0244c8ebd2828fd27019c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61087001c28c4214a2a97c681f874241"
I20260807 20:13:18.712891 4401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81e74d3bc41b4040bbaeb2b07d2ae50d" candidate_uuid: "39e8c651e0244c8ebd2828fd27019c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c49522daf54d36a1f06d5ccd2a14ae"
I20260807 20:13:18.712970 4401 raft_consensus.cc:3037] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:18.712971 4861 raft_consensus.cc:3037] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:18.713752 4861 raft_consensus.cc:2454] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39e8c651e0244c8ebd2828fd27019c4d in term 1.
I20260807 20:13:18.713752 4401 raft_consensus.cc:2454] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39e8c651e0244c8ebd2828fd27019c4d in term 1.
I20260807 20:13:18.713963 4525 leader_election.cc:302] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39e8c651e0244c8ebd2828fd27019c4d, 61087001c28c4214a2a97c681f874241; no voters:
I20260807 20:13:18.714090 5109 raft_consensus.cc:2781] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:18.714147 5109 raft_consensus.cc:686] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [term 1 LEADER]: Becoming Leader. State: Replica: 39e8c651e0244c8ebd2828fd27019c4d, State: Running, Role: LEADER
I20260807 20:13:18.714200 5109 consensus_queue.cc:237] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.714751 4143 catalog_manager.cc:5495] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 39e8c651e0244c8ebd2828fd27019c4d (127.23.200.194). New cstate: current_term: 1 leader_uuid: "39e8c651e0244c8ebd2828fd27019c4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } health_report { overall_health: UNKNOWN } } }
I20260807 20:13:18.727563 4984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.725616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53250 (local address 127.23.200.194:33353)
0807 20:13:18.725659 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:18.725661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:18.725670 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:18.725697 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:18.725698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:18.725707 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:18.725754 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:13:18.725840 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.726330 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.727036 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:18.727219 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:18.727272 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:18.727364 (+ 92us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:13:18.727390 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:13:18.727430 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:18.727483 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 20:13:18.727502 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:13:18.728339 4401 raft_consensus.cc:1261] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Refusing update from remote peer 39e8c651e0244c8ebd2828fd27019c4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:18.728349 4861 raft_consensus.cc:1261] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Refusing update from remote peer 39e8c651e0244c8ebd2828fd27019c4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:18.728613 5109 consensus_queue.cc:1035] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:18.728727 5013 consensus_queue.cc:1035] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:18.731361 4145 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:18.731442 4145 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:18.731972 4145 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a: Sending DeleteTablet for 3 replicas of tablet 81e74d3bc41b4040bbaeb2b07d2ae50d
I20260807 20:13:18.732205 4368 tablet_service.cc:1514] Processing DeleteTablet for tablet 81e74d3bc41b4040bbaeb2b07d2ae50d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:18 UTC) from {username='oryx', principal='oryx/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:43150
I20260807 20:13:18.732221 4828 tablet_service.cc:1514] Processing DeleteTablet for tablet 81e74d3bc41b4040bbaeb2b07d2ae50d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:18 UTC) from {username='oryx', principal='oryx/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:34404
I20260807 20:13:18.732311 4609 tablet_service.cc:1514] Processing DeleteTablet for tablet 81e74d3bc41b4040bbaeb2b07d2ae50d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:18 UTC) from {username='oryx', principal='oryx/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:53226
I20260807 20:13:18.732321 5043 tablet_replica.cc:330] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae: stopping tablet replica
I20260807 20:13:18.732386 5043 raft_consensus.cc:2229] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:18.732410 5044 tablet_replica.cc:330] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d: stopping tablet replica
I20260807 20:13:18.732434 5043 raft_consensus.cc:2258] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:18.732462 5044 raft_consensus.cc:2229] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:13:18.732502 5042 tablet_replica.cc:330] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241: stopping tablet replica
I20260807 20:13:18.732522 5044 raft_consensus.cc:2258] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:18.732566 5042 raft_consensus.cc:2229] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:18.732609 5042 raft_consensus.cc:2258] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:18.732623 4145 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:13:18.732758 4145 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:13:18.732836 5044 ts_tablet_manager.cc:1905] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:18.732895 5043 ts_tablet_manager.cc:1905] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:18.732916 5042 ts_tablet_manager.cc:1905] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:18.733881 4828 tablet_service.cc:1467] Processing CreateTablet for tablet ae0c402160534e4aa39492fee39aefcf (DEFAULT_TABLE table=test-table [id=01226a111b354ef98cdd803c938fcdc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:18.733986 4828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae0c402160534e4aa39492fee39aefcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:18.734674 4368 tablet_service.cc:1467] Processing CreateTablet for tablet ae0c402160534e4aa39492fee39aefcf (DEFAULT_TABLE table=test-table [id=01226a111b354ef98cdd803c938fcdc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:18.734799 5043 ts_tablet_manager.cc:1918] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:13:18.734788 4609 tablet_service.cc:1467] Processing CreateTablet for tablet ae0c402160534e4aa39492fee39aefcf (DEFAULT_TABLE table=test-table [id=01226a111b354ef98cdd803c938fcdc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:18.734824 4368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae0c402160534e4aa39492fee39aefcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:18.734840 5043 log.cc:1196] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-0/wal/wals/81e74d3bc41b4040bbaeb2b07d2ae50d
I20260807 20:13:18.734942 5042 ts_tablet_manager.cc:1918] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:13:18.734987 5042 log.cc:1196] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/81e74d3bc41b4040bbaeb2b07d2ae50d
I20260807 20:13:18.735041 5043 ts_tablet_manager.cc:1939] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 23c49522daf54d36a1f06d5ccd2a14ae: Deleting consensus metadata
I20260807 20:13:18.735183 5042 ts_tablet_manager.cc:1939] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 61087001c28c4214a2a97c681f874241: Deleting consensus metadata
I20260807 20:13:18.735529 5044 ts_tablet_manager.cc:1918] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:13:18.735539 4987 tablet_bootstrap.cc:492] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241: Bootstrap starting.
I20260807 20:13:18.735558 4609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae0c402160534e4aa39492fee39aefcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:18.735581 5044 log.cc:1196] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/81e74d3bc41b4040bbaeb2b07d2ae50d
I20260807 20:13:18.735844 4987 tablet_bootstrap.cc:654] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:18.735899 5044 ts_tablet_manager.cc:1939] T 81e74d3bc41b4040bbaeb2b07d2ae50d P 39e8c651e0244c8ebd2828fd27019c4d: Deleting consensus metadata
I20260807 20:13:18.736088 4995 tablet_bootstrap.cc:492] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae: Bootstrap starting.
I20260807 20:13:18.736392 4995 tablet_bootstrap.cc:654] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:18.736667 4117 catalog_manager.cc:4841] TS 23c49522daf54d36a1f06d5ccd2a14ae (127.23.200.193:42787): tablet 81e74d3bc41b4040bbaeb2b07d2ae50d (table test-table [id=6527bd5a627741c39a83eaf1958eb970]) successfully deleted
I20260807 20:13:18.736704 4989 tablet_bootstrap.cc:492] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d: Bootstrap starting.
I20260807 20:13:18.736780 4122 catalog_manager.cc:4841] TS 39e8c651e0244c8ebd2828fd27019c4d (127.23.200.194:33353): tablet 81e74d3bc41b4040bbaeb2b07d2ae50d (table test-table [id=6527bd5a627741c39a83eaf1958eb970]) successfully deleted
I20260807 20:13:18.736848 4123 catalog_manager.cc:4841] TS 61087001c28c4214a2a97c681f874241 (127.23.200.195:34733): tablet 81e74d3bc41b4040bbaeb2b07d2ae50d (table test-table [id=6527bd5a627741c39a83eaf1958eb970]) successfully deleted
I20260807 20:13:18.737006 4989 tablet_bootstrap.cc:654] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:18.737059 4987 tablet_bootstrap.cc:492] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241: No bootstrap required, opened a new log
I20260807 20:13:18.737128 4987 ts_tablet_manager.cc:1397] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260807 20:13:18.737227 4987 raft_consensus.cc:348] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.737293 4987 raft_consensus.cc:374] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:18.737321 4987 raft_consensus.cc:729] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61087001c28c4214a2a97c681f874241, State: Initialized, Role: FOLLOWER
I20260807 20:13:18.737370 4987 consensus_queue.cc:260] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.737468 4987 ts_tablet_manager.cc:1428] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:18.737527 4989 tablet_bootstrap.cc:492] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d: No bootstrap required, opened a new log
I20260807 20:13:18.737573 4989 ts_tablet_manager.cc:1397] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:18.737680 4989 raft_consensus.cc:348] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.737732 4995 tablet_bootstrap.cc:492] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae: No bootstrap required, opened a new log
I20260807 20:13:18.737743 4989 raft_consensus.cc:374] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:18.737762 4989 raft_consensus.cc:729] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e8c651e0244c8ebd2828fd27019c4d, State: Initialized, Role: FOLLOWER
I20260807 20:13:18.737764 4995 ts_tablet_manager.cc:1397] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260807 20:13:18.737804 4989 consensus_queue.cc:260] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.737890 4989 ts_tablet_manager.cc:1428] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:18.737864 4995 raft_consensus.cc:348] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.737932 4995 raft_consensus.cc:374] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:18.737951 4995 raft_consensus.cc:729] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23c49522daf54d36a1f06d5ccd2a14ae, State: Initialized, Role: FOLLOWER
I20260807 20:13:18.737987 4995 consensus_queue.cc:260] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.738080 4995 ts_tablet_manager.cc:1428] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:18.825176 5109 raft_consensus.cc:482] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:13:18.825282 5109 raft_consensus.cc:504] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.825490 5109 leader_election.cc:288] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23c49522daf54d36a1f06d5ccd2a14ae (127.23.200.193:42787), 61087001c28c4214a2a97c681f874241 (127.23.200.195:34733)
I20260807 20:13:18.825697 4401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0c402160534e4aa39492fee39aefcf" candidate_uuid: "39e8c651e0244c8ebd2828fd27019c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" is_pre_election: true
I20260807 20:13:18.825703 4861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0c402160534e4aa39492fee39aefcf" candidate_uuid: "39e8c651e0244c8ebd2828fd27019c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61087001c28c4214a2a97c681f874241" is_pre_election: true
I20260807 20:13:18.825785 4401 raft_consensus.cc:2454] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39e8c651e0244c8ebd2828fd27019c4d in term 0.
I20260807 20:13:18.825802 4861 raft_consensus.cc:2454] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39e8c651e0244c8ebd2828fd27019c4d in term 0.
I20260807 20:13:18.825996 4525 leader_election.cc:302] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39e8c651e0244c8ebd2828fd27019c4d, 61087001c28c4214a2a97c681f874241; no voters:
I20260807 20:13:18.826100 5109 raft_consensus.cc:2781] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:13:18.826143 5109 raft_consensus.cc:482] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:13:18.826165 5109 raft_consensus.cc:3037] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:18.826853 5109 raft_consensus.cc:504] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.826968 5109 leader_election.cc:288] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [CANDIDATE]: Term 1 election: Requested vote from peers 23c49522daf54d36a1f06d5ccd2a14ae (127.23.200.193:42787), 61087001c28c4214a2a97c681f874241 (127.23.200.195:34733)
I20260807 20:13:18.827116 4401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0c402160534e4aa39492fee39aefcf" candidate_uuid: "39e8c651e0244c8ebd2828fd27019c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c49522daf54d36a1f06d5ccd2a14ae"
I20260807 20:13:18.827148 4861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0c402160534e4aa39492fee39aefcf" candidate_uuid: "39e8c651e0244c8ebd2828fd27019c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61087001c28c4214a2a97c681f874241"
I20260807 20:13:18.827201 4401 raft_consensus.cc:3037] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:18.827214 4861 raft_consensus.cc:3037] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:18.827934 4861 raft_consensus.cc:2454] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39e8c651e0244c8ebd2828fd27019c4d in term 1.
I20260807 20:13:18.827939 4401 raft_consensus.cc:2454] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39e8c651e0244c8ebd2828fd27019c4d in term 1.
I20260807 20:13:18.828131 4525 leader_election.cc:302] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39e8c651e0244c8ebd2828fd27019c4d, 61087001c28c4214a2a97c681f874241; no voters:
I20260807 20:13:18.828236 5109 raft_consensus.cc:2781] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:18.828284 5109 raft_consensus.cc:686] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [term 1 LEADER]: Becoming Leader. State: Replica: 39e8c651e0244c8ebd2828fd27019c4d, State: Running, Role: LEADER
I20260807 20:13:18.828337 5109 consensus_queue.cc:237] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } }
I20260807 20:13:18.828923 4143 catalog_manager.cc:5495] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 39e8c651e0244c8ebd2828fd27019c4d (127.23.200.194). New cstate: current_term: 1 leader_uuid: "39e8c651e0244c8ebd2828fd27019c4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39e8c651e0244c8ebd2828fd27019c4d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 } health_report { overall_health: UNKNOWN } } }
I20260807 20:13:18.834102 4861 raft_consensus.cc:1261] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Refusing update from remote peer 39e8c651e0244c8ebd2828fd27019c4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:18.834280 4401 raft_consensus.cc:1261] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Refusing update from remote peer 39e8c651e0244c8ebd2828fd27019c4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:18.834301 5109 consensus_queue.cc:1035] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "61087001c28c4214a2a97c681f874241" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:18.834462 5109 consensus_queue.cc:1035] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "23c49522daf54d36a1f06d5ccd2a14ae" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:18.843524 4145 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:18.843616 4145 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:18.844177 4145 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 238f29fdf6514a62925344cbe8453b0a: Sending DeleteTablet for 3 replicas of tablet ae0c402160534e4aa39492fee39aefcf
I20260807 20:13:18.844393 4609 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0c402160534e4aa39492fee39aefcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:18 UTC) from {username='oryx', principal='oryx/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:53226
I20260807 20:13:18.844439 4828 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0c402160534e4aa39492fee39aefcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:18 UTC) from {username='oryx', principal='oryx/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:34404
I20260807 20:13:18.844488 5044 tablet_replica.cc:330] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d: stopping tablet replica
I20260807 20:13:18.844514 5042 tablet_replica.cc:330] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241: stopping tablet replica
I20260807 20:13:18.844533 5044 raft_consensus.cc:2229] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:13:18.844515 4368 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0c402160534e4aa39492fee39aefcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:18 UTC) from {username='oryx', principal='oryx/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:43150
I20260807 20:13:18.844556 5042 raft_consensus.cc:2229] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:18.844583 5044 raft_consensus.cc:2258] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:18.844589 5042 raft_consensus.cc:2258] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:18.844619 5043 tablet_replica.cc:330] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae: stopping tablet replica
I20260807 20:13:18.844669 5043 raft_consensus.cc:2229] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:18.844713 5043 raft_consensus.cc:2258] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:18.844760 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 4265
I20260807 20:13:18.844977 5044 ts_tablet_manager.cc:1905] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:18.845032 5043 ts_tablet_manager.cc:1905] T ae0c402160534e4aa39492fee39aefcf P 23c49522daf54d36a1f06d5ccd2a14ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:18.846597 5044 ts_tablet_manager.cc:1918] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:13:18.846663 5044 log.cc:1196] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/ae0c402160534e4aa39492fee39aefcf
I20260807 20:13:18.846825 5044 ts_tablet_manager.cc:1939] T ae0c402160534e4aa39492fee39aefcf P 39e8c651e0244c8ebd2828fd27019c4d: Deleting consensus metadata
I20260807 20:13:18.847132 4122 catalog_manager.cc:4841] TS 39e8c651e0244c8ebd2828fd27019c4d (127.23.200.194:33353): tablet ae0c402160534e4aa39492fee39aefcf (table test-table [id=01226a111b354ef98cdd803c938fcdc0]) successfully deleted
I20260807 20:13:18.856305 5042 ts_tablet_manager.cc:1905] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:18.857728 5042 ts_tablet_manager.cc:1918] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:13:18.857787 5042 log.cc:1196] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/ae0c402160534e4aa39492fee39aefcf
I20260807 20:13:18.857963 5042 ts_tablet_manager.cc:1939] T ae0c402160534e4aa39492fee39aefcf P 61087001c28c4214a2a97c681f874241: Deleting consensus metadata
I20260807 20:13:18.858594 4123 catalog_manager.cc:4841] TS 61087001c28c4214a2a97c681f874241 (127.23.200.195:34733): tablet ae0c402160534e4aa39492fee39aefcf (table test-table [id=01226a111b354ef98cdd803c938fcdc0]) successfully deleted
W20260807 20:13:18.862304 4117 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:42787 (error 108)
I20260807 20:13:18.862749 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 4496
I20260807 20:13:18.862939 5165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:18.862706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:42787 (local address 127.0.0.1:43164)
0807 20:13:18.862811 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:18.862897 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:42787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":37,"threads_started":1}
W20260807 20:13:18.863027 4117 catalog_manager.cc:4584] TS 23c49522daf54d36a1f06d5ccd2a14ae (127.23.200.193:42787): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae0c402160534e4aa39492fee39aefcf: Network error: Client connection negotiation failed: client connection to 127.23.200.193:42787: connect: Connection refused (error 111)
I20260807 20:13:18.874800 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 4733
I20260807 20:13:18.886272 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 4085
2026-08-07T20:13:18Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2900 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:13:18 dist-test-slave-2j7r krb5kdc[5168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:13:18 dist-test-slave-2j7r krb5kdc[5168](info): set up 2 sockets
Aug 07 20:13:18 dist-test-slave-2j7r krb5kdc[5168](info): commencing operation
krb5kdc: starting...
W20260807 20:13:20.043685 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:13:20 dist-test-slave-2j7r krb5kdc[5168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:13:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:13:20.169661 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:41599
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:41599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:20.252699 5198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:20.252930 5198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:20.252962 5198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:20.254201 5198 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:20.254252 5198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:20.254272 5198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:20.254288 5198 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:20.254303 5198 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:13:20.255847 5198 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:41599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:41599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:20.256102 5198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:20.256321 5198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:20.261142 5208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:20.261132 5204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:20.261287 5198 server_base.cc:1034] running on GCE node
W20260807 20:13:20.261134 5205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:20.261837 5198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:20.262125 5198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:20.263269 5198 hybrid_clock.cc:648] HybridClock initialized: now 1786133600263248 us; error 35 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T20:13:20Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1373 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:13:20 dist-test-slave-2j7r krb5kdc[5217](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:13:20 dist-test-slave-2j7r krb5kdc[5217](info): set up 2 sockets
Aug 07 20:13:20 dist-test-slave-2j7r krb5kdc[5217](info): commencing operation
krb5kdc: starting...
W20260807 20:13:21.417796 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:13:21 dist-test-slave-2j7r krb5kdc[5217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:13:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:13:21.540125 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:42395
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:42395
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:21.621860 5244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:21.622068 5244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:21.622097 5244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:21.623226 5244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:21.623268 5244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:21.623286 5244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:21.623301 5244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:21.623317 5244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:13:21.624825 5244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:42395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:42395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:21.625118 5244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:21.625385 5244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:21.630412 5262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:21.630430 5251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:21.630556 5244 server_base.cc:1034] running on GCE node
W20260807 20:13:21.630659 5250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:21.630875 5244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:21.631167 5244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:21.632359 5244 hybrid_clock.cc:648] HybridClock initialized: now 1786133601632336 us; error 39 us; skew 500 ppm
Aug 07 20:13:21 dist-test-slave-2j7r krb5kdc[5217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:21.638991 5244 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:13:21.639416 5244 webserver.cc:460] Webserver started at http://127.23.200.254:42641/ using document root <none> and password file <none>
I20260807 20:13:21.639614 5244 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:21.639654 5244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:21.639745 5244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:21.640758 5244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "9917795898144082a625d88603661b06"
format_stamp: "Formatted at 2026-08-07 20:13:21 on dist-test-slave-2j7r"
server_key: "ab011e08f47573da2d9e2c58caaae51d"
server_key_iv: "0d2c059883cfb31686f9c41bf06c404c"
server_key_version: "encryptionkey@0"
I20260807 20:13:21.641043 5244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "9917795898144082a625d88603661b06"
format_stamp: "Formatted at 2026-08-07 20:13:21 on dist-test-slave-2j7r"
server_key: "ab011e08f47573da2d9e2c58caaae51d"
server_key_iv: "0d2c059883cfb31686f9c41bf06c404c"
server_key_version: "encryptionkey@0"
I20260807 20:13:21.642388 5244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:21.643095 5278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:21.643256 5244 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:21.643342 5244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "9917795898144082a625d88603661b06"
format_stamp: "Formatted at 2026-08-07 20:13:21 on dist-test-slave-2j7r"
server_key: "ab011e08f47573da2d9e2c58caaae51d"
server_key_iv: "0d2c059883cfb31686f9c41bf06c404c"
server_key_version: "encryptionkey@0"
I20260807 20:13:21.643397 5244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:21.657910 5244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:21.660024 5244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:21.660141 5244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:21.663861 5244 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:42395
I20260807 20:13:21.663894 5361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:42395 every 8 connection(s)
I20260807 20:13:21.664239 5244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:13:21.665059 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 5244
I20260807 20:13:21.665155 5364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:21.665184 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:13:21.665360 24355 external_mini_cluster.cc:1428] Setting key 812b3422de5f59f007b40672e080cf37
I20260807 20:13:21.667892 5364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06: Bootstrap starting.
I20260807 20:13:21.668494 5364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:21.668744 5364 log.cc:824] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:21.669379 5364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06: No bootstrap required, opened a new log
I20260807 20:13:21.671217 5364 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917795898144082a625d88603661b06" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42395 } }
I20260807 20:13:21.671387 5364 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Aug 07 20:13:21 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:21.671430 5364 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9917795898144082a625d88603661b06, State: Initialized, Role: FOLLOWER
I20260807 20:13:21.671582 5364 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917795898144082a625d88603661b06" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42395 } }
I20260807 20:13:21.671653 5364 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:13:21.671684 5364 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:13:21.671731 5364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:21.672240 5364 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917795898144082a625d88603661b06" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42395 } }
I20260807 20:13:21.672338 5364 leader_election.cc:302] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9917795898144082a625d88603661b06; no voters:
I20260807 20:13:21.672508 5364 leader_election.cc:288] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:13:21.672565 5371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:21.672763 5364 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:21.672977 5371 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 1 LEADER]: Becoming Leader. State: Replica: 9917795898144082a625d88603661b06, State: Running, Role: LEADER
I20260807 20:13:21.673133 5371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917795898144082a625d88603661b06" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42395 } }
I20260807 20:13:21.673552 5373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9917795898144082a625d88603661b06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917795898144082a625d88603661b06" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42395 } } }
I20260807 20:13:21.673583 5374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9917795898144082a625d88603661b06. Latest consensus state: current_term: 1 leader_uuid: "9917795898144082a625d88603661b06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917795898144082a625d88603661b06" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42395 } } }
I20260807 20:13:21.673635 5373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:21.673657 5374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:21.673825 5385 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:13:21.674365 5385 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:21.675582 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:21.665880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39640 (local address 127.23.200.254:42395)
0807 20:13:21.666068 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:21.666077 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:21.666101 (+ 24us) server_negotiation.cc:407] Connection header received
0807 20:13:21.666553 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:21.666560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:21.666606 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:21.666731 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:21.667321 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:21.668231 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:21.669026 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:21.669489 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:21.669597 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:21.671959 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:21.671964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:21.671964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:21.671976 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:21.673814 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:21.674229 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:21.674232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:21.674233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:21.674331 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:21.674552 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:21.674553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:21.674553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:21.674845 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:21.674991 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:21.675137 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 20:13:21.675272 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":95,"threads_started":1}
I20260807 20:13:21.676339 5385 catalog_manager.cc:1347] Generated new cluster ID: 5818f94df1e648feb247f5396b0cd610
I20260807 20:13:21.676383 5385 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:13:21.681181 5385 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:13:21.681878 5385 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:21.697327 5385 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06: Generated new TSK 0
I20260807 20:13:21.697552 5385 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:13:21.728331 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:42395
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:21.810959 5417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:21.811163 5417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:21.811194 5417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:21.812341 5417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:21.812384 5417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:21.812426 5417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:13:21.813910 5417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:21.814139 5417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:21.814348 5417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:21.819332 5425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:21.819340 5432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:21.819387 5417 server_base.cc:1034] running on GCE node
W20260807 20:13:21.819327 5429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:21.819687 5417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:21.819936 5417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:21.821106 5417 hybrid_clock.cc:648] HybridClock initialized: now 1786133601821073 us; error 29 us; skew 500 ppm
Aug 07 20:13:21 dist-test-slave-2j7r krb5kdc[5217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:21.823513 5417 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:13:21.823941 5417 webserver.cc:460] Webserver started at http://127.23.200.193:43549/ using document root <none> and password file <none>
I20260807 20:13:21.824141 5417 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:21.824188 5417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:21.824316 5417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:21.825433 5417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "6261802f4655444dbc0bfc971947f15b"
format_stamp: "Formatted at 2026-08-07 20:13:21 on dist-test-slave-2j7r"
server_key: "5c1fb5ab00551ea8fa19808e29af1578"
server_key_iv: "17bac6abdc9e2eb644b545b371936c15"
server_key_version: "encryptionkey@0"
I20260807 20:13:21.825768 5417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "6261802f4655444dbc0bfc971947f15b"
format_stamp: "Formatted at 2026-08-07 20:13:21 on dist-test-slave-2j7r"
server_key: "5c1fb5ab00551ea8fa19808e29af1578"
server_key_iv: "17bac6abdc9e2eb644b545b371936c15"
server_key_version: "encryptionkey@0"
I20260807 20:13:21.827188 5417 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:13:21.827877 5440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:21.828027 5417 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:21.828112 5417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "6261802f4655444dbc0bfc971947f15b"
format_stamp: "Formatted at 2026-08-07 20:13:21 on dist-test-slave-2j7r"
server_key: "5c1fb5ab00551ea8fa19808e29af1578"
server_key_iv: "17bac6abdc9e2eb644b545b371936c15"
server_key_version: "encryptionkey@0"
I20260807 20:13:21.828171 5417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:21.847473 5417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:21.849578 5417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:21.849718 5417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:21.849947 5417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:21.850253 5417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:21.850286 5417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:21.850314 5417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:21.850332 5417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:21.855928 5417 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:37601
I20260807 20:13:21.856014 5646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:37601 every 8 connection(s)
I20260807 20:13:21.856336 5417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
Aug 07 20:13:21 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:21.864182 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 5417
I20260807 20:13:21.864291 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:13:21.864529 24355 external_mini_cluster.cc:1428] Setting key 76359f812a7f3482d033aaa403853f52
I20260807 20:13:21.864777 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:21.857029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:48161 (local address 127.23.200.254:42395)
0807 20:13:21.857127 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:21.857130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:21.857282 (+ 152us) server_negotiation.cc:407] Connection header received
0807 20:13:21.857778 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:21.857779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:21.857790 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:21.857845 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:21.858736 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:21.859235 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:21.860152 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:21.860357 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:21.860407 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:21.862422 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:21.862424 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:21.862424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:21.862431 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:21.863523 (+ 1092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:21.863856 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:21.863857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:21.863857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:21.863907 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:21.864117 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:21.864118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:21.864118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:21.864255 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:21.864325 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:21.864617 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 20:13:21.864709 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 20:13:21.864974 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:21.856989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:48161)
0807 20:13:21.857160 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:21.857202 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:21.857674 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:21.857923 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:21.857926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:21.858470 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:21.858664 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:21.858669 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:21.859296 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:21.859297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:21.860082 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:21.860084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:21.860416 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:21.860417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:21.860508 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:21.860970 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:21.860979 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:21.862338 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:21.863590 (+ 1252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:21.863597 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:21.863598 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:21.863801 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:21.863968 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:21.863969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:21.863969 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:21.864064 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:21.864326 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:21.864333 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:21.864481 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 20:13:21.864576 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 20:13:21.864740 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260807 20:13:21.865434 5647 heartbeater.cc:347] Connected to a master server at 127.23.200.254:42395
I20260807 20:13:21.865541 5647 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:21.865722 5647 heartbeater.cc:511] Master 127.23.200.254:42395 requested a full tablet report, sending...
I20260807 20:13:21.866216 5309 ts_manager.cc:194] Registered new tserver with Master: 6261802f4655444dbc0bfc971947f15b (127.23.200.193:37601)
I20260807 20:13:21.866809 5309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:48161
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:13:21.890180 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:42395
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:21.972295 5654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:21.972539 5654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:21.972568 5654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:21.973773 5654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:21.973814 5654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:21.973838 5654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:13:21.975342 5654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:21.975543 5654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:21.975725 5654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:21.980690 5663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:21.980786 5654 server_base.cc:1034] running on GCE node
W20260807 20:13:21.980669 5665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:21.980913 5660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:21.981132 5654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:21.981374 5654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:21.982517 5654 hybrid_clock.cc:648] HybridClock initialized: now 1786133601982506 us; error 28 us; skew 500 ppm
Aug 07 20:13:21 dist-test-slave-2j7r krb5kdc[5217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:21.984894 5654 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:13:21.985339 5654 webserver.cc:460] Webserver started at http://127.23.200.194:34341/ using document root <none> and password file <none>
I20260807 20:13:21.985582 5654 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:21.985631 5654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:21.985752 5654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:21.986780 5654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "812bf22c23d747818e94ef49713fef4c"
format_stamp: "Formatted at 2026-08-07 20:13:21 on dist-test-slave-2j7r"
server_key: "1eec24c8b889c8d6f674aa1c0d6781a2"
server_key_iv: "dfe6dc3994e1ba4461451080151611b7"
server_key_version: "encryptionkey@0"
I20260807 20:13:21.987107 5654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "812bf22c23d747818e94ef49713fef4c"
format_stamp: "Formatted at 2026-08-07 20:13:21 on dist-test-slave-2j7r"
server_key: "1eec24c8b889c8d6f674aa1c0d6781a2"
server_key_iv: "dfe6dc3994e1ba4461451080151611b7"
server_key_version: "encryptionkey@0"
I20260807 20:13:21.988301 5654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:13:21.988932 5677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:21.989100 5654 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:13:21.989192 5654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "812bf22c23d747818e94ef49713fef4c"
format_stamp: "Formatted at 2026-08-07 20:13:21 on dist-test-slave-2j7r"
server_key: "1eec24c8b889c8d6f674aa1c0d6781a2"
server_key_iv: "dfe6dc3994e1ba4461451080151611b7"
server_key_version: "encryptionkey@0"
I20260807 20:13:21.989255 5654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:22.009500 5654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:22.011343 5654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:22.011462 5654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:22.011646 5654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:22.011914 5654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:22.011945 5654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:22.011966 5654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:22.011983 5654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:22.017683 5654 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:38733
I20260807 20:13:22.017742 5874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:38733 every 8 connection(s)
I20260807 20:13:22.018061 5654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133601, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:22.025456 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 5654
I20260807 20:13:22.025563 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:13:22.025784 24355 external_mini_cluster.cc:1428] Setting key 34c60ee292a3e2fcdc5e8036274dab88
I20260807 20:13:22.026625 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:22.018734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:33431 (local address 127.23.200.254:42395)
0807 20:13:22.018835 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:22.018837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:22.018970 (+ 133us) server_negotiation.cc:407] Connection header received
0807 20:13:22.019456 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:22.019457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:22.019467 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:22.019522 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:22.020445 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.020935 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.021923 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.022115 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.022170 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.024087 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:22.024088 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:22.024089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:22.024095 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:22.025203 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:22.025634 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:22.025635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:22.025636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:22.025696 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:22.025945 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:22.025946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:22.025946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:22.026096 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:22.026160 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:22.026456 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 20:13:22.026554 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 20:13:22.026793 5880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:22.018691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:33431)
0807 20:13:22.018856 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:22.018899 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:22.019361 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:22.019586 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:22.019589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:22.020146 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:22.020341 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.020346 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.021008 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.021009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.021848 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.021850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.022177 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.022178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.022277 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.022740 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:22.022749 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:22.024012 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:22.025263 (+ 1251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:22.025272 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:22.025274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:22.025569 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:22.025753 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:22.025753 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:22.025754 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:22.025889 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:22.026155 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:22.026162 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:22.026294 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 20:13:22.026386 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 20:13:22.026566 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260807 20:13:22.027216 5876 heartbeater.cc:347] Connected to a master server at 127.23.200.254:42395
I20260807 20:13:22.027313 5876 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:22.027508 5876 heartbeater.cc:511] Master 127.23.200.254:42395 requested a full tablet report, sending...
I20260807 20:13:22.027873 5309 ts_manager.cc:194] Registered new tserver with Master: 812bf22c23d747818e94ef49713fef4c (127.23.200.194:38733)
I20260807 20:13:22.028230 5309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:33431
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:13:22.051446 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:42395
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:22.135054 5883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:22.135268 5883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:22.135300 5883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:22.136479 5883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:22.136525 5883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:22.136560 5883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:13:22.138010 5883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:22.138249 5883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:22.138545 5883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:22.143611 5902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:22.143617 5891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:22.143618 5896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:22.143674 5883 server_base.cc:1034] running on GCE node
I20260807 20:13:22.143982 5883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:22.144202 5883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:22.145341 5883 hybrid_clock.cc:648] HybridClock initialized: now 1786133602145327 us; error 31 us; skew 500 ppm
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133602, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:22.147910 5883 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:13:22.148350 5883 webserver.cc:460] Webserver started at http://127.23.200.195:43435/ using document root <none> and password file <none>
I20260807 20:13:22.148530 5883 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:22.148567 5883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:22.148660 5883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:22.149685 5883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "8f1dab30a10b45e1854c07f934c9fe22"
format_stamp: "Formatted at 2026-08-07 20:13:22 on dist-test-slave-2j7r"
server_key: "019437eb1270b83150453848720b02d1"
server_key_iv: "7d1b384fea09a10fa8df88311b775b15"
server_key_version: "encryptionkey@0"
I20260807 20:13:22.149991 5883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "8f1dab30a10b45e1854c07f934c9fe22"
format_stamp: "Formatted at 2026-08-07 20:13:22 on dist-test-slave-2j7r"
server_key: "019437eb1270b83150453848720b02d1"
server_key_iv: "7d1b384fea09a10fa8df88311b775b15"
server_key_version: "encryptionkey@0"
I20260807 20:13:22.151191 5883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:13:22.151873 5909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:22.152065 5883 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:22.152155 5883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "8f1dab30a10b45e1854c07f934c9fe22"
format_stamp: "Formatted at 2026-08-07 20:13:22 on dist-test-slave-2j7r"
server_key: "019437eb1270b83150453848720b02d1"
server_key_iv: "7d1b384fea09a10fa8df88311b775b15"
server_key_version: "encryptionkey@0"
I20260807 20:13:22.152217 5883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:22.158993 5883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:22.160748 5883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:22.160871 5883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:22.161149 5883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:22.161458 5883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:22.161496 5883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:22.161518 5883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:22.161541 5883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:22.166996 5883 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:38471
I20260807 20:13:22.167062 6105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:38471 every 8 connection(s)
I20260807 20:13:22.167331 5883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133602, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:22.176120 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:22.168017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:37377 (local address 127.23.200.254:42395)
0807 20:13:22.168057 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:22.168059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:22.168261 (+ 202us) server_negotiation.cc:407] Connection header received
0807 20:13:22.168769 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:22.168771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:22.168785 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:22.168838 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:22.169832 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.170342 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.171294 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.171483 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.171544 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.173606 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:22.173608 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:22.173608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:22.173615 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:22.174970 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:22.175302 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:22.175302 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:22.175303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:22.175346 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:22.175581 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:22.175581 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:22.175581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:22.175673 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:22.175719 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:22.175982 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 20:13:22.176054 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 20:13:22.176206 6108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:22.167977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:37377)
0807 20:13:22.168143 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:22.168178 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:22.168668 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:22.168913 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:22.168918 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:22.169538 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:22.169731 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.169736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.170435 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.170436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.171209 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.171212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.171557 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.171558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.171676 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.172077 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:22.172086 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:22.173530 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:22.175038 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:22.175044 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:22.175047 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:22.175247 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:22.175396 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:22.175396 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:22.175397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:22.175526 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:22.175722 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:22.175727 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:22.175841 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 20:13:22.175928 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 20:13:22.176044 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260807 20:13:22.176595 6106 heartbeater.cc:347] Connected to a master server at 127.23.200.254:42395
I20260807 20:13:22.176681 6106 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:22.176800 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 5883
I20260807 20:13:22.176853 6106 heartbeater.cc:511] Master 127.23.200.254:42395 requested a full tablet report, sending...
I20260807 20:13:22.176885 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:13:22.177060 24355 external_mini_cluster.cc:1428] Setting key 2bbe1dc1385a921b7a6f1262582128fb
I20260807 20:13:22.177280 5309 ts_manager.cc:194] Registered new tserver with Master: 8f1dab30a10b45e1854c07f934c9fe22 (127.23.200.195:38471)
I20260807 20:13:22.177781 5309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:37377
I20260807 20:13:22.177992 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.193.keytab.
Entry for principal oryx/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.193.keytab.
WARNING: no policy specified for oryx/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.194.keytab.
Entry for principal oryx/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.194.keytab.
WARNING: no policy specified for oryx/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.195.keytab.
Entry for principal oryx/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.195.keytab.
I20260807 20:13:22.251705 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 5417
I20260807 20:13:22.282106 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 5654
I20260807 20:13:22.295644 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 5883
I20260807 20:13:22.309283 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 5244
I20260807 20:13:22.322291 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:42395
--webserver_interface=127.23.200.254
--webserver_port=42641
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:42395
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:22.406812 6123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:22.407047 6123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:22.407070 6123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:22.408296 6123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:22.408347 6123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:22.408360 6123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:22.408371 6123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:22.408382 6123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:13:22.409938 6123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:42395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:42395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=42641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:22.410188 6123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:22.410429 6123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:22.415493 6131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:22.415488 6129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:22.415519 6123 server_base.cc:1034] running on GCE node
W20260807 20:13:22.415485 6135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:22.415933 6123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:22.416215 6123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:22.417356 6123 hybrid_clock.cc:648] HybridClock initialized: now 1786133602417337 us; error 30 us; skew 500 ppm
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133602, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:22.419961 6123 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:13:22.420390 6123 webserver.cc:460] Webserver started at http://127.23.200.254:42641/ using document root <none> and password file <none>
I20260807 20:13:22.420624 6123 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:22.420696 6123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:22.422127 6123 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:22.422809 6150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:22.422976 6123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:22.423065 6123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "9917795898144082a625d88603661b06"
format_stamp: "Formatted at 2026-08-07 20:13:21 on dist-test-slave-2j7r"
server_key: "ab011e08f47573da2d9e2c58caaae51d"
server_key_iv: "0d2c059883cfb31686f9c41bf06c404c"
server_key_version: "encryptionkey@0"
I20260807 20:13:22.423396 6123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:22.441238 6123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:22.443408 6123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:22.443565 6123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:22.447561 6123 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:42395
I20260807 20:13:22.447597 6244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:42395 every 8 connection(s)
I20260807 20:13:22.447993 6123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:13:22.449594 6246 sys_catalog.cc:263] Verifying existing consensus state
I20260807 20:13:22.450248 6246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06: Bootstrap starting.
I20260807 20:13:22.457878 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 6123
I20260807 20:13:22.458195 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:37601
--local_ip_for_outbound_sockets=127.23.200.193
--tserver_master_addrs=127.23.200.254:42395
--webserver_port=43549
--webserver_interface=127.23.200.193
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.200.193
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:13:22.458431 6246 log.cc:824] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:22.460250 6246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:13:22.460575 6246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06: Bootstrap complete.
I20260807 20:13:22.462708 6246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917795898144082a625d88603661b06" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42395 } }
I20260807 20:13:22.462936 6246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9917795898144082a625d88603661b06, State: Initialized, Role: FOLLOWER
I20260807 20:13:22.463101 6246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917795898144082a625d88603661b06" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42395 } }
I20260807 20:13:22.463181 6246 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:13:22.463232 6246 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:13:22.463281 6246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 1 FOLLOWER]: Advancing to term 2
I20260807 20:13:22.464278 6246 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917795898144082a625d88603661b06" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42395 } }
I20260807 20:13:22.464376 6246 leader_election.cc:302] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9917795898144082a625d88603661b06; no voters:
I20260807 20:13:22.464571 6246 leader_election.cc:288] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 20:13:22.464629 6256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 20:13:22.464834 6256 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [term 2 LEADER]: Becoming Leader. State: Replica: 9917795898144082a625d88603661b06, State: Running, Role: LEADER
I20260807 20:13:22.464850 6246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:22.465011 6256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917795898144082a625d88603661b06" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42395 } }
I20260807 20:13:22.465432 6258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9917795898144082a625d88603661b06. Latest consensus state: current_term: 2 leader_uuid: "9917795898144082a625d88603661b06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917795898144082a625d88603661b06" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42395 } } }
I20260807 20:13:22.465531 6258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:22.465885 6257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9917795898144082a625d88603661b06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9917795898144082a625d88603661b06" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42395 } } }
I20260807 20:13:22.465986 6257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:22.466109 6274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:13:22.466677 6274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:22.466807 6274 catalog_manager.cc:1259] Loaded cluster ID: 5818f94df1e648feb247f5396b0cd610
I20260807 20:13:22.466841 6274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:13:22.468004 6274 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:22.468182 6274 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9917795898144082a625d88603661b06: Loaded TSK: 0
I20260807 20:13:22.468441 6274 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 20:13:22.544993 6253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:22.545269 6253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:22.545306 6253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:22.546578 6253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:22.546627 6253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:22.546666 6253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:13:22.548228 6253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.193.keytab
--principal=oryx/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:37601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=43549
--tserver_master_addrs=127.23.200.254:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:22.548496 6253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:22.548727 6253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:22.554008 6301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:22.554106 6253 server_base.cc:1034] running on GCE node
W20260807 20:13:22.554016 6300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:22.554270 6308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:22.554488 6253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:22.554821 6253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:22.555997 6253 hybrid_clock.cc:648] HybridClock initialized: now 1786133602555984 us; error 32 us; skew 500 ppm
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133602, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:22.559329 6253 init.cc:377] Logged in from keytab as oryx/127.23.200.193@KRBTEST.COM (short username oryx)
I20260807 20:13:22.559908 6253 webserver.cc:460] Webserver started at http://127.23.200.193:43549/ using document root <none> and password file <none>
I20260807 20:13:22.560199 6253 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:22.560278 6253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:22.562135 6253 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:13:22.562820 6319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:22.563019 6253 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:13:22.563124 6253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "6261802f4655444dbc0bfc971947f15b"
format_stamp: "Formatted at 2026-08-07 20:13:21 on dist-test-slave-2j7r"
server_key: "5c1fb5ab00551ea8fa19808e29af1578"
server_key_iv: "17bac6abdc9e2eb644b545b371936c15"
server_key_version: "encryptionkey@0"
I20260807 20:13:22.563580 6253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:22.583015 6253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:22.585440 6253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:22.585587 6253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:22.585829 6253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:22.586189 6253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:22.586229 6253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:22.586263 6253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:22.586285 6253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:22.593380 6253 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:37601
I20260807 20:13:22.593417 6525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:37601 every 8 connection(s)
I20260807 20:13:22.593864 6253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:13:22.594260 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 6253
I20260807 20:13:22.594599 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:38733
--local_ip_for_outbound_sockets=127.23.200.194
--tserver_master_addrs=127.23.200.254:42395
--webserver_port=34341
--webserver_interface=127.23.200.194
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.200.194
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:22.605943 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:22.594856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:46987)
0807 20:13:22.595070 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:22.595124 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:22.595739 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:22.597739 (+ 2000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:22.597743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:22.598499 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:22.598772 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.598779 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.600321 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.600322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.601646 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.601650 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.602397 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.602398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.602542 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.603098 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:22.603111 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:22.605708 (+ 2597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":108,"threads_started":1}
W20260807 20:13:22.606004 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:22.595229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:46987 (local address 127.23.200.254:42395)
0807 20:13:22.597362 (+ 2133us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:22.597372 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:22.597404 (+ 32us) server_negotiation.cc:407] Connection header received
0807 20:13:22.597516 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:22.597521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:22.597563 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:22.597685 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:22.598928 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.600201 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.601741 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.602305 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.602395 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.605824 (+ 3429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:46987: BlockingRecv error: recv got EOF from 127.23.200.193:46987 (error 108)
Metrics: {"server-negotiator.queue_time_us":2073,"thread_start_us":85,"threads_started":1}
W20260807 20:13:22.606220 6526 heartbeater.cc:644] Failed to heartbeat to 127.23.200.254:42395 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.200.254:42395: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:22.610736 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:22.606461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:50791)
0807 20:13:22.606513 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:22.606522 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:22.606557 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:22.606684 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:22.606685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:22.606824 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:22.606874 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.606876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.607435 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.607436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.608263 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.608265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.608558 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.608559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.608622 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.608980 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:22.608985 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:22.610672 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 20:13:22.610816 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:22.606509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:50791 (local address 127.23.200.254:42395)
0807 20:13:22.606550 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:22.606552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:22.606561 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:22.606613 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:22.606614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:22.606624 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:22.606669 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:22.606934 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.607376 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.608326 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.608504 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.608547 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.610766 (+ 2219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:50791: BlockingRecv error: recv got EOF from 127.23.200.193:50791 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:22.615252 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:22.611062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:33079)
0807 20:13:22.611112 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:22.611123 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:22.611166 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:22.611285 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:22.611286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:22.611418 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:22.611449 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.611451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.612011 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.612012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.612761 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.612764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.613071 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.613072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.613156 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.613510 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:22.613515 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:22.615191 (+ 1676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 20:13:22.615377 6526 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 20:13:22.615374 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:22.611110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:33079 (local address 127.23.200.254:42395)
0807 20:13:22.611154 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:22.611155 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:22.611164 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:22.611221 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:22.611222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:22.611230 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:22.611270 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:22.611510 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.611940 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.612833 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.613014 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.613060 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.615274 (+ 2214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:33079: BlockingRecv error: recv got EOF from 127.23.200.193:33079 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 20:13:22.683058 6530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:22.683265 6530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:22.683290 6530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:22.684465 6530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:22.684509 6530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:22.684535 6530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:13:22.686033 6530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.194.keytab
--principal=oryx/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:38733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=34341
--tserver_master_addrs=127.23.200.254:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:22.686256 6530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:22.686488 6530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:22.691489 6550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:22.691526 6530 server_base.cc:1034] running on GCE node
W20260807 20:13:22.691526 6546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:22.691495 6545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:22.691875 6530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:22.692123 6530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:22.693289 6530 hybrid_clock.cc:648] HybridClock initialized: now 1786133602693268 us; error 33 us; skew 500 ppm
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133602, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:22.695964 6530 init.cc:377] Logged in from keytab as oryx/127.23.200.194@KRBTEST.COM (short username oryx)
I20260807 20:13:22.696400 6530 webserver.cc:460] Webserver started at http://127.23.200.194:34341/ using document root <none> and password file <none>
I20260807 20:13:22.696635 6530 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:22.696698 6530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:22.698045 6530 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:22.698624 6564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:22.698778 6530 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:22.698863 6530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "812bf22c23d747818e94ef49713fef4c"
format_stamp: "Formatted at 2026-08-07 20:13:21 on dist-test-slave-2j7r"
server_key: "1eec24c8b889c8d6f674aa1c0d6781a2"
server_key_iv: "dfe6dc3994e1ba4461451080151611b7"
server_key_version: "encryptionkey@0"
I20260807 20:13:22.699189 6530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:22.713943 6530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:22.715694 6530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:22.715832 6530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:22.716073 6530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:22.716392 6530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:22.716429 6530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:22.716459 6530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:22.716477 6530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:22.722087 6530 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:38733
I20260807 20:13:22.722143 6745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:38733 every 8 connection(s)
I20260807 20:13:22.722507 6530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 20:13:22.729650 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:22.723249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:40687 (local address 127.23.200.254:42395)
0807 20:13:22.723330 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:22.723333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:22.723571 (+ 238us) server_negotiation.cc:407] Connection header received
0807 20:13:22.724054 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:22.724056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:22.724070 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:22.724125 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:22.725071 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.725628 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.726626 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.726853 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.726930 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.729599 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:40687: BlockingRecv error: recv got EOF from 127.23.200.194:40687 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 20:13:22.729662 6750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:22.723228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:40687)
0807 20:13:22.723451 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:22.723488 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:22.723972 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:22.724180 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:22.724184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:22.724742 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:22.724949 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.724954 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.725711 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.725712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.726557 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.726560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.726937 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.726938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.727037 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.727470 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:22.727478 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:22.729504 (+ 2026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":147,"threads_started":1}
W20260807 20:13:22.729871 6746 heartbeater.cc:644] Failed to heartbeat to 127.23.200.254:42395 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.200.254:42395: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
I20260807 20:13:22.731592 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 6530
I20260807 20:13:22.731972 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:38471
--local_ip_for_outbound_sockets=127.23.200.195
--tserver_master_addrs=127.23.200.254:42395
--webserver_port=43435
--webserver_interface=127.23.200.195
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.200.195
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:22.734341 6750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:22.730102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:38561)
0807 20:13:22.730169 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:22.730181 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:22.730227 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:22.730376 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:22.730377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:22.730498 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:22.730546 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.730547 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.731152 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.731153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.731817 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.731820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.732221 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.732222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.732272 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.732672 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:22.732675 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:22.734275 (+ 1600us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 20:13:22.734407 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:22.730225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:38561 (local address 127.23.200.254:42395)
0807 20:13:22.730282 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:22.730283 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:22.730290 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:13:22.730310 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:22.730310 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:22.730318 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:22.730358 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:22.730595 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.731068 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.731896 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.732108 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.732168 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.734351 (+ 2183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:38561: BlockingRecv error: recv got EOF from 127.23.200.194:38561 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:22.738948 6750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:22.734669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:46721)
0807 20:13:22.734770 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:22.734780 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:22.734824 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:22.734953 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:22.734954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:22.735071 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:22.735097 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.735098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.735864 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.735865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.736321 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.736322 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.736895 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.736896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.736940 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.737322 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:22.737326 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:22.738897 (+ 1571us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":84}
W20260807 20:13:22.739055 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:22.734729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:46721 (local address 127.23.200.254:42395)
0807 20:13:22.734773 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:22.734775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:22.734810 (+ 35us) server_negotiation.cc:407] Connection header received
0807 20:13:22.734875 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:22.734876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:22.734885 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:22.734935 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:22.735161 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.735796 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.736394 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.736824 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.736895 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.738998 (+ 2103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:46721: BlockingRecv error: recv got EOF from 127.23.200.194:46721 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260807 20:13:22.739142 6746 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 20:13:22.816752 6753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:22.816962 6753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:22.816994 6753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:22.818184 6753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:22.818236 6753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:22.818271 6753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:13:22.819695 6753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/krb5kdc/oryx_127.23.200.195.keytab
--principal=oryx/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:38471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=43435
--tserver_master_addrs=127.23.200.254:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:22.819937 6753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:22.820163 6753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:22.825284 6771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:22.825256 6768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:22.825383 6764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:22.825417 6753 server_base.cc:1034] running on GCE node
I20260807 20:13:22.825723 6753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:22.825970 6753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:22.827152 6753 hybrid_clock.cc:648] HybridClock initialized: now 1786133602827134 us; error 27 us; skew 500 ppm
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133602, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:22.829723 6753 init.cc:377] Logged in from keytab as oryx/127.23.200.195@KRBTEST.COM (short username oryx)
I20260807 20:13:22.830194 6753 webserver.cc:460] Webserver started at http://127.23.200.195:43435/ using document root <none> and password file <none>
I20260807 20:13:22.830428 6753 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:22.830490 6753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:22.831841 6753 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:22.832563 6780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:22.832746 6753 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:22.832834 6753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "8f1dab30a10b45e1854c07f934c9fe22"
format_stamp: "Formatted at 2026-08-07 20:13:22 on dist-test-slave-2j7r"
server_key: "019437eb1270b83150453848720b02d1"
server_key_iv: "7d1b384fea09a10fa8df88311b775b15"
server_key_version: "encryptionkey@0"
I20260807 20:13:22.833212 6753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:22.842103 6753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:22.844131 6753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:22.844245 6753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:22.844442 6753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:22.844723 6753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:22.844758 6753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:22.844779 6753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:22.844791 6753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:22.850447 6753 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:38471
I20260807 20:13:22.850519 6953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:38471 every 8 connection(s)
I20260807 20:13:22.850829 6753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:22.857946 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:22.851569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:59053)
0807 20:13:22.851750 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:22.851789 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:22.852266 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:22.852491 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:22.852495 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:22.853060 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:22.853313 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.853319 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.853956 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.853956 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.854737 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.854739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.855067 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.855068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.855173 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.855671 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:22.855679 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:22.857790 (+ 2111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":98,"threads_started":1}
W20260807 20:13:22.857986 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:22.851599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:59053 (local address 127.23.200.254:42395)
0807 20:13:22.851692 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:22.851694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:22.851872 (+ 178us) server_negotiation.cc:407] Connection header received
0807 20:13:22.852353 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:22.852355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:22.852366 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:22.852429 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:22.853386 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.853872 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.854808 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.855000 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.855048 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.857893 (+ 2845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:59053: BlockingRecv error: recv got EOF from 127.23.200.195:59053 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260807 20:13:22.858263 6956 heartbeater.cc:644] Failed to heartbeat to 127.23.200.254:42395 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.200.254:42395: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
I20260807 20:13:22.858351 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 6753
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:22.863139 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:22.858498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:41037)
0807 20:13:22.858530 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:22.858540 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:22.858580 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:22.858841 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:22.858843 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:22.859003 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:22.859064 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.859066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.859676 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.859677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.860207 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.860209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.860514 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.860514 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.860580 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.860970 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:22.860975 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:22.863067 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":15}
W20260807 20:13:22.863273 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:22.858675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:41037 (local address 127.23.200.254:42395)
0807 20:13:22.858733 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:22.858735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:22.858743 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:13:22.858764 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:22.858765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:22.858776 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:22.858827 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:22.859130 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.859600 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.860276 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.860454 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.860501 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.863188 (+ 2687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:41037: BlockingRecv error: recv got EOF from 127.23.200.195:41037 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:13:22.866573 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:22.858720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39648 (local address 127.23.200.254:42395)
0807 20:13:22.858818 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:22.858820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:22.858973 (+ 153us) server_negotiation.cc:407] Connection header received
0807 20:13:22.859039 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:22.859040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:22.859047 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:22.859089 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:22.859624 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.860069 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.860914 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.861147 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.861201 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.862378 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:22.862383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:22.862384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:22.862396 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:22.864374 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:22.864777 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:22.864781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:22.864782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:22.864906 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:22.865185 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:22.865187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:22.865187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:22.865653 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:22.865851 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:22.866364 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 20:13:22.866482 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":54,"threads_started":1}
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:22 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:22.869431 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:22.863476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:34537)
0807 20:13:22.863598 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:22.863607 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:22.863645 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:22.863763 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:22.863764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:22.863950 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:22.863985 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.863987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.864864 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.864865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.865666 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:22.865668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.867014 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.867016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:22.867090 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.867442 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:22.867446 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:22.869355 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109}
W20260807 20:13:22.869510 6529 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:22.863563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:34537 (local address 127.23.200.254:42395)
0807 20:13:22.863660 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:22.863662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:22.863669 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:13:22.863687 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:22.863688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:22.863712 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:22.863758 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:22.864087 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.864781 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.865760 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:22.866028 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:22.866094 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:22.869454 (+ 3360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:34537: BlockingRecv error: recv got EOF from 127.23.200.195:34537 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 20:13:22.869799 6956 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 20:13:23 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:23 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:23.620358 6967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:23.615777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:44719)
0807 20:13:23.615948 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:23.615959 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:23.616018 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:23.616152 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:23.616152 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:23.616320 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:23.616371 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:23.616373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:23.616987 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:23.616987 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:23.617558 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:23.617560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:23.617976 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:23.617977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:23.618030 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:23.618523 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:23.618528 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:23.620286 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":83,"threads_started":1}
W20260807 20:13:23.620515 6968 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:23.615894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:44719 (local address 127.23.200.254:42395)
0807 20:13:23.616009 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:23.616011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:23.616020 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:23.616067 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:23.616068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:23.616078 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:23.616135 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:23.616427 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:23.616925 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:23.617647 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:23.617873 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:23.617947 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:23.620433 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:44719: BlockingRecv error: recv got EOF from 127.23.200.193:44719 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
Aug 07 20:13:23 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:23 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:23.744069 6970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:23.739564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:53825)
0807 20:13:23.739686 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:23.739697 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:23.739758 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:23.739871 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:23.739872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:23.740040 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:23.740092 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:23.740094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:23.740691 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:23.740691 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:23.741488 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:23.741493 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:23.741798 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:23.741798 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:23.741852 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:23.742249 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:23.742253 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:23.744002 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
W20260807 20:13:23.744158 6968 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:23.739625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:53825 (local address 127.23.200.254:42395)
0807 20:13:23.739684 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:23.739686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:23.739751 (+ 65us) server_negotiation.cc:407] Connection header received
0807 20:13:23.739802 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:23.739803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:23.739813 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:23.739861 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:23.740140 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:23.740636 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:23.741556 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:23.741744 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:23.741794 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:23.744099 (+ 2305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:53825: BlockingRecv error: recv got EOF from 127.23.200.194:53825 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 07 20:13:23 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:23 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:23.874747 6972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:23.870249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:54309)
0807 20:13:23.870398 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:23.870410 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:23.870474 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:23.870605 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:23.870606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:23.870768 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:23.870820 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:23.870821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:23.871451 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:23.871453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:23.872123 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:23.872126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:23.872461 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:23.872462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:23.872502 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:23.872877 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:23.872881 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:23.874675 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
W20260807 20:13:23.874828 6968 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:23.870376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:54309 (local address 127.23.200.254:42395)
0807 20:13:23.870439 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:23.870441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:23.870456 (+ 15us) server_negotiation.cc:407] Connection header received
0807 20:13:23.870533 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:23.870534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:23.870544 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:23.870591 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:23.870869 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:23.871356 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:23.872203 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:23.872393 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:23.872446 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:23.874768 (+ 2322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:54309: BlockingRecv error: recv got EOF from 127.23.200.195:54309 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 07 20:13:24 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:24 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:24.625861 6975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:24.620890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:34351)
0807 20:13:24.621047 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:24.621058 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:24.621145 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:24.621295 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:24.621296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:24.621472 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:24.621521 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:24.621523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:24.622239 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:24.622240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:24.622816 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:24.622818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:24.623223 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:24.623224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:24.623275 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:24.623744 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:24.623749 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:24.625749 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
W20260807 20:13:24.625946 6976 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:24.621004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:34351 (local address 127.23.200.254:42395)
0807 20:13:24.621144 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:24.621146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:24.621156 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:13:24.621205 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:24.621205 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:24.621216 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:24.621281 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:24.621626 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:24.622147 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:24.622911 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:24.623140 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:24.623208 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:24.625872 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:34351: BlockingRecv error: recv got EOF from 127.23.200.193:34351 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":57,"threads_started":1}
Aug 07 20:13:24 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:24 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:24.749431 6977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:24.744576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:56435)
0807 20:13:24.744748 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:24.744763 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:24.744852 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:24.745013 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:24.745015 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:24.745285 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:24.745343 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:24.745346 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:24.746081 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:24.746082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:24.746612 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:24.746614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:24.746966 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:24.746967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:24.747025 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:24.747485 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:24.747490 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:24.749360 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":70,"threads_started":1}
W20260807 20:13:24.749565 6976 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:24.744688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:56435 (local address 127.23.200.254:42395)
0807 20:13:24.744759 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:24.744761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:24.744835 (+ 74us) server_negotiation.cc:407] Connection header received
0807 20:13:24.744934 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:24.744935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:24.744948 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:24.745000 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:24.745438 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:24.745995 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:24.746694 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:24.746898 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:24.746962 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:24.749493 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:56435: BlockingRecv error: recv got EOF from 127.23.200.194:56435 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 20:13:24 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:24 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:24.879640 6978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:24.875204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:47783)
0807 20:13:24.875329 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:24.875340 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:24.875412 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:24.875552 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:24.875553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:24.875719 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:24.875768 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:24.875770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:24.876556 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:24.876557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:24.877108 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:24.877111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:24.877442 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:24.877443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:24.877499 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:24.877919 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:24.877925 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:24.879564 (+ 1639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":58,"threads_started":1}
W20260807 20:13:24.879714 6976 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:24.875337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:47783 (local address 127.23.200.254:42395)
0807 20:13:24.875393 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:24.875395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:24.875402 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:13:24.875476 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:24.875477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:24.875492 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:24.875540 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:24.875836 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:24.876479 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:24.877174 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:24.877378 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:24.877433 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:24.879662 (+ 2229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:47783: BlockingRecv error: recv got EOF from 127.23.200.195:47783 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 07 20:13:25 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:25 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:25.630967 6980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:25.626434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:40721)
0807 20:13:25.626555 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:25.626566 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:25.626621 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:25.626769 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:25.626770 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:25.626952 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:25.627002 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:25.627004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:25.627626 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:25.627627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:25.628169 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:25.628170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:25.628557 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:25.628558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:25.628605 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:25.629057 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:25.629062 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:25.630908 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
W20260807 20:13:25.631094 6982 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:25.626516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:40721 (local address 127.23.200.254:42395)
0807 20:13:25.626659 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:25.626662 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:25.626669 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:13:25.626691 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:25.626692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:25.626702 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:25.626761 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:25.627057 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:25.627564 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:25.628265 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:25.628479 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:25.628542 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:25.631023 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:40721: BlockingRecv error: recv got EOF from 127.23.200.193:40721 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
Aug 07 20:13:25 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:25 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:25.754547 6987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:25.749889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:48779)
0807 20:13:25.750079 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:25.750091 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:25.750163 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:25.750301 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:25.750302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:25.750529 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:25.750578 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:25.750581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:25.751253 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:25.751254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:25.751783 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:25.751785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:25.752125 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:25.752126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:25.752181 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:25.752621 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:25.752627 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:25.754440 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":103,"threads_started":1}
W20260807 20:13:25.754606 6982 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:25.749988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:48779 (local address 127.23.200.254:42395)
0807 20:13:25.750058 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:25.750060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:25.750153 (+ 93us) server_negotiation.cc:407] Connection header received
0807 20:13:25.750231 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:25.750232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:25.750245 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:25.750295 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:25.750655 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:25.751176 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:25.751861 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:25.752062 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:25.752118 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:25.754542 (+ 2424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:48779: BlockingRecv error: recv got EOF from 127.23.200.194:48779 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 20:13:25 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:25 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:25.884685 6989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:25.880145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:43653)
0807 20:13:25.880281 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:25.880292 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:25.880353 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:25.880488 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:25.880489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:25.880680 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:25.880732 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:25.880734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:25.881407 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:25.881408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:25.881953 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:25.881955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:25.882281 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:25.882283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:25.882326 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:25.882796 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:25.882800 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:25.884554 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
W20260807 20:13:25.884725 6982 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:25.880263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:43653 (local address 127.23.200.254:42395)
0807 20:13:25.880324 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:25.880326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:25.880337 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:13:25.880410 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:25.880411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:25.880423 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:25.880469 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:25.880809 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:25.881335 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:25.882025 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:25.882209 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:25.882261 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:25.884661 (+ 2400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:43653: BlockingRecv error: recv got EOF from 127.23.200.195:43653 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 07 20:13:26 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:26 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:26.636202 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:26.631443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:60165)
0807 20:13:26.631613 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:26.631624 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:26.631680 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:26.631832 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:26.631833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:26.632006 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:26.632055 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:26.632057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:26.632699 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:26.632700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:26.633272 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:26.633274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:26.633717 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:26.633718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:26.633761 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:26.634214 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:26.634219 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:26.636124 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":66,"threads_started":1}
W20260807 20:13:26.636276 6991 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:26.631539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:60165 (local address 127.23.200.254:42395)
0807 20:13:26.631654 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:26.631656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:26.631666 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:13:26.631741 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:26.631742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:26.631752 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:26.631809 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:26.632112 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:26.632624 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:26.633430 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:26.633636 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:26.633701 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:26.636217 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:60165: BlockingRecv error: recv got EOF from 127.23.200.193:60165 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
Aug 07 20:13:26 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:26 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:26.763453 6994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:26.755031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:34979)
0807 20:13:26.755167 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:26.755179 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:26.755253 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:26.755412 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:26.755413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:26.755579 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:26.755628 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:26.755630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:26.756305 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:26.756306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:26.756850 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:26.756853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:26.757231 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:26.757232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:26.757287 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:26.761554 (+ 4267us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:26.761560 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:26.763364 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
W20260807 20:13:26.763533 6991 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:26.755109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:34979 (local address 127.23.200.254:42395)
0807 20:13:26.755158 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:26.755160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:26.755251 (+ 91us) server_negotiation.cc:407] Connection header received
0807 20:13:26.755317 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:26.755318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:26.755330 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:26.755382 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:26.755702 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:26.756229 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:26.756937 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:26.757175 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:26.757232 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:26.763479 (+ 6247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:34979: BlockingRecv error: recv got EOF from 127.23.200.194:34979 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 07 20:13:26 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:26 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:26.889463 6995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:26.885203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:45975)
0807 20:13:26.885323 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:26.885332 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:26.885393 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:26.885506 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:26.885507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:26.885668 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:26.885718 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:26.885720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:26.886344 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:26.886344 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:26.886852 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:26.886853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:26.887159 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:26.887160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:26.887197 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:26.887634 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:26.887639 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:26.889374 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
W20260807 20:13:26.889523 6991 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:26.885247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:45975 (local address 127.23.200.254:42395)
0807 20:13:26.885292 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:26.885294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:26.885374 (+ 80us) server_negotiation.cc:407] Connection header received
0807 20:13:26.885436 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:26.885436 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:26.885449 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:26.885493 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:26.885770 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:26.886285 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:26.886925 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:26.887107 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:26.887155 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:26.889471 (+ 2316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:45975: BlockingRecv error: recv got EOF from 127.23.200.195:45975 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 07 20:13:27 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:27 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:27.642148 6996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:27.636830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:57657)
0807 20:13:27.637014 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:27.637028 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:27.637123 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:27.637285 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:27.637286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:27.637467 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:27.637518 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:27.637520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:27.638451 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:27.638453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:27.639038 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:27.639041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:27.639466 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:27.639468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:27.639521 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:27.640020 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:27.640026 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:27.642083 (+ 2057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":80,"threads_started":1}
W20260807 20:13:27.642259 6998 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:27.636945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:57657 (local address 127.23.200.254:42395)
0807 20:13:27.637139 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:27.637141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:27.637171 (+ 30us) server_negotiation.cc:407] Connection header received
0807 20:13:27.637196 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:27.637196 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:27.637208 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:27.637257 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:27.637636 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:27.638319 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:27.639163 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:27.639375 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:27.639441 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:27.642188 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:57657: BlockingRecv error: recv got EOF from 127.23.200.193:57657 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":75,"threads_started":1}
Aug 07 20:13:27 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:27 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:27.769016 6999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:27.763981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:38389)
0807 20:13:27.764113 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:27.764123 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:27.764170 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:27.764340 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:27.764341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:27.764512 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:27.764571 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:27.764573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:27.765342 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:27.765344 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:27.765965 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:27.765968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:27.766375 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:27.766376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:27.766423 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:27.766917 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:27.766922 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:27.768937 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":59,"threads_started":1}
W20260807 20:13:27.769119 6998 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:27.764121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:38389 (local address 127.23.200.254:42395)
0807 20:13:27.764183 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:27.764185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:27.764194 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:27.764220 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:27.764221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:27.764231 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:27.764286 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:27.764642 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:27.765220 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:27.766038 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:27.766250 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:27.766354 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:27.769045 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:38389: BlockingRecv error: recv got EOF from 127.23.200.194:38389 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 07 20:13:27 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:27 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:27.894384 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:27.889976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:58067)
0807 20:13:27.890104 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:27.890117 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:27.890182 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:27.890331 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:27.890332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:27.890510 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:27.890567 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:27.890568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:27.891254 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:27.891256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:27.891777 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:27.891779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:27.892120 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:27.892120 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:27.892157 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:27.892556 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:27.892560 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:27.894318 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
W20260807 20:13:27.894479 6998 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:27.890056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:58067 (local address 127.23.200.254:42395)
0807 20:13:27.890161 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:27.890163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:27.890173 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:13:27.890243 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:27.890243 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:27.890257 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:27.890315 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:27.890657 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:27.891161 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:27.891866 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:27.892064 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:27.892116 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:27.894415 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:58067: BlockingRecv error: recv got EOF from 127.23.200.195:58067 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 20:13:28 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:28 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:28.647827 7003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:28.642696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:43489)
0807 20:13:28.642835 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:28.642847 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:28.642918 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:28.643091 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:28.643092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:28.643383 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:28.643448 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:28.643451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:28.644183 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:28.644185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:28.644744 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:28.644747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:28.645378 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:28.645379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:28.645431 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:28.645849 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:28.645853 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:28.647766 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
W20260807 20:13:28.647921 7004 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:28.642766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:43489 (local address 127.23.200.254:42395)
0807 20:13:28.642938 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:28.642940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:28.642950 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:13:28.642977 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:28.642978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:28.642991 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:28.643067 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:28.643524 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:28.644084 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:28.644848 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:28.645062 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:28.645156 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:28.647852 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:43489: BlockingRecv error: recv got EOF from 127.23.200.193:43489 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
Aug 07 20:13:28 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:28 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:28.774417 7006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:28.769582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:48329)
0807 20:13:28.769735 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:28.769745 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:28.769811 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:28.769935 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:28.769936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:28.770100 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:28.770164 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:28.770165 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:28.770801 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:28.770803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:28.771394 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:28.771397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:28.771751 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:28.771753 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:28.771812 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:28.772218 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:28.772265 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:28.774352 (+ 2087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
W20260807 20:13:28.774545 7004 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:28.769656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:48329 (local address 127.23.200.254:42395)
0807 20:13:28.769725 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:28.769727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:28.769809 (+ 82us) server_negotiation.cc:407] Connection header received
0807 20:13:28.769869 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:28.769870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:28.769880 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:28.769922 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:28.770213 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:28.770706 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:28.771478 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:28.771673 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:28.771724 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:28.774474 (+ 2750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:48329: BlockingRecv error: recv got EOF from 127.23.200.194:48329 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 07 20:13:28 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:28 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:28.899430 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:28.894846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:37783)
0807 20:13:28.895008 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:28.895019 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:28.895084 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:28.895201 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:28.895202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:28.895421 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:28.895472 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:28.895474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:28.896312 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:28.896313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:28.896850 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:28.896852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:28.897192 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:28.897194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:28.897292 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:28.897705 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:28.897709 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:28.899371 (+ 1662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
W20260807 20:13:28.899518 7004 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:28.894914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:37783 (local address 127.23.200.254:42395)
0807 20:13:28.894974 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:28.894977 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:28.895075 (+ 98us) server_negotiation.cc:407] Connection header received
0807 20:13:28.895127 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:28.895128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:28.895140 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:28.895188 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:28.895531 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:28.896198 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:28.896929 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:28.897132 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:28.897188 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:28.899458 (+ 2270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:37783: BlockingRecv error: recv got EOF from 127.23.200.195:37783 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 07 20:13:29 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:29 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:29.653375 7009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:29.648314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:58831)
0807 20:13:29.648447 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:29.648457 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:29.648508 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:29.648705 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:29.648706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:29.648875 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:29.648924 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:29.648925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:29.649760 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:29.649762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:29.650339 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:29.650342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:29.650779 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:29.650780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:29.650836 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:29.651290 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:29.651295 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:29.653312 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
W20260807 20:13:29.653422 7020 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:29.648439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:58831 (local address 127.23.200.254:42395)
0807 20:13:29.648583 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:29.648585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:29.648593 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:13:29.648616 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:29.648617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:29.648627 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:29.648695 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:29.648978 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:29.649683 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:29.650453 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:29.650687 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:29.650764 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:29.653363 (+ 2599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:58831: BlockingRecv error: recv got EOF from 127.23.200.193:58831 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
Aug 07 20:13:29 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:29 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:29.779539 7021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:29.774927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:45877)
0807 20:13:29.775081 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:29.775094 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:29.775177 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:29.775320 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:29.775321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:29.775514 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:29.775563 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:29.775565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:29.776214 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:29.776215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:29.776739 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:29.776741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:29.777074 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:29.777075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:29.777155 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:29.777598 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:29.777603 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:29.779461 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
W20260807 20:13:29.779615 7020 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:29.775036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:45877 (local address 127.23.200.254:42395)
0807 20:13:29.775100 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:29.775102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:29.775155 (+ 53us) server_negotiation.cc:407] Connection header received
0807 20:13:29.775244 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:29.775245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:29.775255 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:29.775308 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:29.775645 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:29.776140 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:29.776816 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:29.777011 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:29.777066 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:29.779562 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:45877: BlockingRecv error: recv got EOF from 127.23.200.194:45877 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 20:13:29 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:29 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:29.904278 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:29.899897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:59671)
0807 20:13:29.900069 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:29.900081 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:29.900147 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:29.900264 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:29.900271 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:29.900432 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:29.900481 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:29.900483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:29.901126 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:29.901127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:29.901724 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:29.901726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:29.902053 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:29.902054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:29.902101 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:29.902561 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:29.902566 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:29.904199 (+ 1633us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
W20260807 20:13:29.904354 7020 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:29.900017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:59671 (local address 127.23.200.254:42395)
0807 20:13:29.900082 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:29.900084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:29.900139 (+ 55us) server_negotiation.cc:407] Connection header received
0807 20:13:29.900201 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:29.900202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:29.900214 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:29.900250 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:29.900532 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:29.901034 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:29.901788 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:29.901996 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:29.902046 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:29.904302 (+ 2256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:59671: BlockingRecv error: recv got EOF from 127.23.200.195:59671 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 07 20:13:30 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:30 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:30.658654 7026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:30.653959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:60933)
0807 20:13:30.654126 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:30.654139 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:30.654216 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:30.654447 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:30.654449 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:30.654626 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:30.654677 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:30.654678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:30.655317 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:30.655318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:30.655873 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:30.655875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:30.656205 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:30.656207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:30.656261 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:30.656686 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:30.656691 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:30.658591 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":70,"threads_started":1}
W20260807 20:13:30.658771 7027 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:30.654122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:60933 (local address 127.23.200.254:42395)
0807 20:13:30.654255 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:30.654258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:30.654265 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:13:30.654331 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:30.654332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:30.654342 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:30.654406 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:30.654754 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:30.655258 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:30.655934 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:30.656120 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:30.656167 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:30.658690 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:60933: BlockingRecv error: recv got EOF from 127.23.200.193:60933 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
Aug 07 20:13:30 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:30 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:30.784502 7028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:30.780014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:58451)
0807 20:13:30.780167 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:30.780178 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:30.780237 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:30.780391 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:30.780392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:30.780554 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:30.780605 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:30.780607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:30.781394 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:30.781395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:30.781938 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:30.781940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:30.782243 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:30.782244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:30.782286 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:30.782691 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:30.782695 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:30.784422 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":65,"threads_started":1}
W20260807 20:13:30.784577 7027 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:30.780099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:58451 (local address 127.23.200.254:42395)
0807 20:13:30.780159 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:30.780167 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:30.780278 (+ 111us) server_negotiation.cc:407] Connection header received
0807 20:13:30.780306 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:30.780307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:30.780321 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:30.780363 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:30.780667 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:30.781320 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:30.781995 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:30.782187 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:30.782233 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:30.784525 (+ 2292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:58451: BlockingRecv error: recv got EOF from 127.23.200.194:58451 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 07 20:13:30 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:30 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:30.909449 7029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:30.904812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:36845)
0807 20:13:30.904945 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:30.904956 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:30.905025 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:30.905241 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:30.905243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:30.905423 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:30.905474 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:30.905476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:30.906143 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:30.906144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:30.906719 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:30.906721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:30.907055 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:30.907056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:30.907113 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:30.907532 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:30.907537 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:30.909385 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
W20260807 20:13:30.909546 7027 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:30.904858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:36845 (local address 127.23.200.254:42395)
0807 20:13:30.904913 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:30.904916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:30.905014 (+ 98us) server_negotiation.cc:407] Connection header received
0807 20:13:30.905156 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:30.905157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:30.905172 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:30.905222 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:30.905543 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:30.906068 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:30.906797 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:30.906995 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:30.907050 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:30.909485 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:36845: BlockingRecv error: recv got EOF from 127.23.200.195:36845 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 07 20:13:31 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:31 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:31.663650 7030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:31.659128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:55877)
0807 20:13:31.659265 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:31.659276 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:31.659338 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:31.659489 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:31.659490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:31.659657 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:31.659718 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:31.659720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:31.660387 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:31.660388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:31.660930 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:31.660932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:31.661355 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:31.661356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:31.661392 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:31.661808 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:31.661812 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:31.663587 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
W20260807 20:13:31.663761 7031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:31.659232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:55877 (local address 127.23.200.254:42395)
0807 20:13:31.659375 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:31.659377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:31.659385 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:13:31.659407 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:31.659407 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:31.659418 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:31.659475 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:31.659779 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:31.660327 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:31.661028 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:31.661294 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:31.661358 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:31.663685 (+ 2327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:55877: BlockingRecv error: recv got EOF from 127.23.200.193:55877 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":55,"threads_started":1}
Aug 07 20:13:31 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:31 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:31.794355 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:31.785105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:45109)
0807 20:13:31.785251 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:31.785314 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:31.785389 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:31.785548 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:31.785550 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:31.785724 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:31.785780 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:31.785782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:31.786546 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:31.786547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:31.787066 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:31.787068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:31.787371 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:31.787373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:31.787426 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:31.792305 (+ 4879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:31.792311 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:31.794242 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
W20260807 20:13:31.794425 7031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:31.785204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:45109 (local address 127.23.200.254:42395)
0807 20:13:31.785282 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:31.785284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:31.785382 (+ 98us) server_negotiation.cc:407] Connection header received
0807 20:13:31.785435 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:31.785436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:31.785449 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:31.785501 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:31.785872 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:31.786480 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:31.787127 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:31.787309 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:31.787352 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:31.794351 (+ 6999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:45109: BlockingRecv error: recv got EOF from 127.23.200.194:45109 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 20:13:31 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:31 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:31.914485 7035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:31.909941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:38523)
0807 20:13:31.910095 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:31.910108 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:31.910180 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:31.910307 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:31.910309 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:31.910532 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:31.910584 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:31.910586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:31.911259 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:31.911260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:31.911789 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:31.911791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:31.912121 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:31.912122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:31.912178 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:31.912607 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:31.912612 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:31.914377 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
W20260807 20:13:31.914525 7031 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:31.910039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:38523 (local address 127.23.200.254:42395)
0807 20:13:31.910111 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:31.910112 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:31.910169 (+ 57us) server_negotiation.cc:407] Connection header received
0807 20:13:31.910237 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:31.910238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:31.910251 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:31.910301 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:31.910663 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:31.911184 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:31.911864 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:31.912061 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:31.912114 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:31.914473 (+ 2359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:38523: BlockingRecv error: recv got EOF from 127.23.200.195:38523 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 20:13:32 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:32 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:32.669070 7036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:32.664122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:38817)
0807 20:13:32.664257 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:32.664267 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:32.664318 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:32.664486 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:32.664487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:32.664657 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:32.664708 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:32.664709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:32.665538 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:32.665539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:32.666146 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:32.666149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:32.666545 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:32.666546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:32.666588 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:32.667057 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:32.667062 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:32.668973 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
W20260807 20:13:32.669147 7037 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:32.664230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:38817 (local address 127.23.200.254:42395)
0807 20:13:32.664358 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:32.664360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:32.664369 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:32.664397 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:32.664397 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:32.664408 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:32.664471 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:32.664785 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:32.665459 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:32.666241 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:32.666487 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:32.666541 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:32.669058 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:38817: BlockingRecv error: recv got EOF from 127.23.200.193:38817 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
Aug 07 20:13:32 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:32 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:32.799531 7038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:32.794969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:47631)
0807 20:13:32.795150 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:32.795162 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:32.795233 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:32.795375 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:32.795376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:32.795549 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:32.795598 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:32.795599 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:32.796255 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:32.796256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:32.796782 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:32.796784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:32.797150 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:32.797151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:32.797196 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:32.797665 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:32.797670 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:32.799469 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":97,"threads_started":1}
W20260807 20:13:32.799669 7037 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:32.795095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:47631 (local address 127.23.200.254:42395)
0807 20:13:32.795150 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:32.795152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:32.795232 (+ 80us) server_negotiation.cc:407] Connection header received
0807 20:13:32.795289 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:32.795290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:32.795301 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:32.795349 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:32.795681 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:32.796181 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:32.796869 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:32.797066 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:32.797147 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:32.799603 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:47631: BlockingRecv error: recv got EOF from 127.23.200.194:47631 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 07 20:13:32 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:32 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:32.919552 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:32.915011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:37485)
0807 20:13:32.915164 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:32.915174 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:32.915238 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:32.915383 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:32.915385 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:32.915552 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:32.915603 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:32.915604 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:32.916239 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:32.916241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:32.916777 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:32.916779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:32.917144 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:32.917146 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:32.917207 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:32.917658 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:32.917663 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:32.919471 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":62,"threads_started":1}
W20260807 20:13:32.919636 7037 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:32.915144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:37485 (local address 127.23.200.254:42395)
0807 20:13:32.915199 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:32.915201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:32.915223 (+ 22us) server_negotiation.cc:407] Connection header received
0807 20:13:32.915306 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:32.915307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:32.915319 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:32.915375 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:32.915654 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:32.916169 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:32.916833 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:32.917031 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:32.917079 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:32.919578 (+ 2499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:37485: BlockingRecv error: recv got EOF from 127.23.200.195:37485 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 07 20:13:33 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:33 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:33.674430 7041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:33.669626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:33159)
0807 20:13:33.669778 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:33.669789 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:33.669850 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:33.670041 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:33.670042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:33.670214 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:33.670320 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:33.670322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:33.670999 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:33.671000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:33.671555 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:33.671557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:33.671885 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:33.671887 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:33.671936 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:33.672356 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:33.672361 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:33.674359 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":63,"threads_started":1}
W20260807 20:13:33.674515 7042 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:33.669691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:33159 (local address 127.23.200.254:42395)
0807 20:13:33.669808 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:33.669810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:33.669837 (+ 27us) server_negotiation.cc:407] Connection header received
0807 20:13:33.669922 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:33.669922 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:33.669934 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:33.670010 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:33.670381 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:33.670885 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:33.671623 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:33.671818 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:33.671864 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:33.674451 (+ 2587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:33159: BlockingRecv error: recv got EOF from 127.23.200.193:33159 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
Aug 07 20:13:33 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:33 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:33.804870 7043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:33.800145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:45685)
0807 20:13:33.800332 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:33.800347 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:33.800423 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:33.800549 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:33.800550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:33.800732 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:33.800786 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:33.800787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:33.801520 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:33.801521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:33.802063 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:33.802065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:33.802458 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:33.802460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:33.802507 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:33.802925 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:33.802930 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:33.804794 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":78,"threads_started":1}
W20260807 20:13:33.804976 7042 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:33.800318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:45685 (local address 127.23.200.254:42395)
0807 20:13:33.800377 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:33.800379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:33.800412 (+ 33us) server_negotiation.cc:407] Connection header received
0807 20:13:33.800482 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:33.800483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:33.800494 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:33.800532 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:33.800836 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:33.801430 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:33.802141 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:33.802356 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:33.802442 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:33.804906 (+ 2464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:45685: BlockingRecv error: recv got EOF from 127.23.200.194:45685 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 07 20:13:33 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:33 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:33.925225 7047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:33.920272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:37493)
0807 20:13:33.920450 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:33.920462 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:33.920534 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:33.920793 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:33.920795 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:33.920987 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:33.921038 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:33.921040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:33.921764 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:33.921765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:33.922295 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:33.922297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:33.922692 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:33.922693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:33.922755 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:33.923196 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:33.923201 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:33.925131 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":106,"threads_started":1}
W20260807 20:13:33.925259 7042 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:33.920327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:37493 (local address 127.23.200.254:42395)
0807 20:13:33.920419 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:33.920421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:33.920584 (+ 163us) server_negotiation.cc:407] Connection header received
0807 20:13:33.920616 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:33.920617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:33.920632 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:33.920693 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:33.921137 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:33.921684 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:33.922393 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:33.922601 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:33.922668 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:33.925202 (+ 2534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:37493: BlockingRecv error: recv got EOF from 127.23.200.195:37493 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 20:13:34 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:34 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:34.679581 7048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:34.674945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:58745)
0807 20:13:34.675072 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:34.675084 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:34.675146 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:34.675293 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:34.675293 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:34.675502 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:34.675551 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:34.675553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:34.676201 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:34.676202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:34.676739 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:34.676742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:34.677161 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:34.677163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:34.677204 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:34.677634 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:34.677639 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:34.679520 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
W20260807 20:13:34.679723 7049 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:34.675053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:58745 (local address 127.23.200.254:42395)
0807 20:13:34.675168 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:34.675170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:34.675179 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:34.675202 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:34.675203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:34.675213 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:34.675279 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:34.675629 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:34.676142 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:34.676829 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:34.677038 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:34.677151 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:34.679637 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:58745: BlockingRecv error: recv got EOF from 127.23.200.193:58745 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":52,"threads_started":1}
Aug 07 20:13:34 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:34 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:34.809864 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:34.805350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:60063)
0807 20:13:34.805530 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:34.805542 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:34.805608 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:34.805734 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:34.805735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:34.805934 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:34.805996 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:34.805998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:34.806643 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:34.806645 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:34.807251 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:34.807254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:34.807565 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:34.807565 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:34.807609 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:34.808044 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:34.808049 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:34.809787 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":129,"threads_started":1}
W20260807 20:13:34.809984 7049 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:34.805438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:60063 (local address 127.23.200.254:42395)
0807 20:13:34.805495 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:34.805497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:34.805589 (+ 92us) server_negotiation.cc:407] Connection header received
0807 20:13:34.805666 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:34.805667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:34.805679 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:34.805718 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:34.806055 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:34.806560 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:34.807324 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:34.807512 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:34.807556 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:34.809889 (+ 2333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:60063: BlockingRecv error: recv got EOF from 127.23.200.194:60063 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 07 20:13:34 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:34 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:34.930717 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:34.925817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:47279)
0807 20:13:34.925970 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:34.925981 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:34.926050 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:34.926270 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:34.926272 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:34.926466 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:34.926523 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:34.926525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:34.927420 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:34.927421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:34.927999 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:34.928001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:34.928328 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:34.928330 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:34.928381 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:34.928834 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:34.928839 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:34.930639 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
W20260807 20:13:34.930828 7049 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:34.925906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:47279 (local address 127.23.200.254:42395)
0807 20:13:34.925979 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:34.925981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:34.926041 (+ 60us) server_negotiation.cc:407] Connection header received
0807 20:13:34.926106 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:34.926107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:34.926119 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:34.926168 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:34.926630 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:34.927333 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:34.928063 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:34.928250 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:34.928294 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:34.930758 (+ 2464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:47279: BlockingRecv error: recv got EOF from 127.23.200.195:47279 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 07 20:13:35 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:35 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:35.685191 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:35.680097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:45527)
0807 20:13:35.680249 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:35.680260 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:35.680371 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:35.680559 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:35.680560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:35.680744 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:35.680794 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:35.680796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:35.681579 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:35.681581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:35.682154 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:35.682157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:35.682592 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:35.682593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:35.682644 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:35.683091 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:35.683096 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:35.685073 (+ 1977us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":67,"threads_started":1}
W20260807 20:13:35.685256 7058 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:35.680206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:45527 (local address 127.23.200.254:42395)
0807 20:13:35.680326 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:35.680328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:35.680355 (+ 27us) server_negotiation.cc:407] Connection header received
0807 20:13:35.680426 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:35.680426 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:35.680437 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:35.680503 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:35.680888 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:35.681466 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:35.682287 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:35.682502 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:35.682555 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:35.685196 (+ 2641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:45527: BlockingRecv error: recv got EOF from 127.23.200.193:45527 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
Aug 07 20:13:35 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:35 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:35.815201 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:35.810411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:55003)
0807 20:13:35.810557 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:35.810567 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:35.810619 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:35.810780 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:35.810786 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:35.810960 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:35.811009 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:35.811010 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:35.811746 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:35.811748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:35.812311 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:35.812314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:35.812732 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:35.812733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:35.812781 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:35.813239 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:35.813244 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:35.815132 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":64,"threads_started":1}
W20260807 20:13:35.815289 7058 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:35.810579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:55003 (local address 127.23.200.254:42395)
0807 20:13:35.810639 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:35.810641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:35.810650 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:35.810679 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:35.810680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:35.810692 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:35.810760 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:35.811090 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:35.811660 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:35.812445 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:35.812657 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:35.812727 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:35.815225 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:55003: BlockingRecv error: recv got EOF from 127.23.200.194:55003 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 07 20:13:35 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:35 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:35.935694 7060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:35.931261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:49365)
0807 20:13:35.931395 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:35.931415 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:35.931486 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:35.931621 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:35.931623 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:35.931806 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:35.931855 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:35.931857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:35.932544 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:35.932546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:35.933071 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:35.933073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:35.933431 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:35.933432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:35.933487 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:35.933911 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:35.933915 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:35.935617 (+ 1702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
W20260807 20:13:35.935766 7058 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:35.931366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:49365 (local address 127.23.200.254:42395)
0807 20:13:35.931438 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:35.931440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:35.931465 (+ 25us) server_negotiation.cc:407] Connection header received
0807 20:13:35.931544 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:35.931545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:35.931560 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:35.931607 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:35.931963 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:35.932464 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:35.933172 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:35.933363 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:35.933419 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:35.935711 (+ 2292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:49365: BlockingRecv error: recv got EOF from 127.23.200.195:49365 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 20:13:36 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:36 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:36.690591 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:36.685687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:49693)
0807 20:13:36.685876 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:36.685889 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:36.685962 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:36.686119 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:36.686120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:36.686311 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:36.686362 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:36.686364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:36.687144 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:36.687145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:36.687710 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:36.687712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:36.688100 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:36.688102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:36.688166 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:36.688688 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:36.688693 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:36.690530 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":137,"threads_started":1}
W20260807 20:13:36.690716 7063 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:36.685773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:49693 (local address 127.23.200.254:42395)
0807 20:13:36.685890 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:36.685892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:36.685938 (+ 46us) server_negotiation.cc:407] Connection header received
0807 20:13:36.686005 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:36.686006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:36.686016 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:36.686076 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:36.686465 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:36.687059 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:36.687798 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:36.688014 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:36.688066 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:36.690642 (+ 2576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:49693: BlockingRecv error: recv got EOF from 127.23.200.193:49693 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":56,"threads_started":1}
Aug 07 20:13:36 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:36 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:36.823729 7066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:36.815848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:59103)
0807 20:13:36.815993 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:36.816006 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:36.816078 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:36.816188 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:36.816190 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:36.816423 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:36.816473 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:36.816475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:36.817181 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:36.817183 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:36.817745 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:36.817747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:36.818171 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:36.818171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:36.818207 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:36.821873 (+ 3666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:36.821878 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:36.823665 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":58,"threads_started":1}
W20260807 20:13:36.823879 7063 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:36.815963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:59103 (local address 127.23.200.254:42395)
0807 20:13:36.816022 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:36.816024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:36.816053 (+ 29us) server_negotiation.cc:407] Connection header received
0807 20:13:36.816131 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:36.816132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:36.816142 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:36.816182 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:36.816542 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:36.817048 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:36.817897 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:36.818101 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:36.818175 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:36.823806 (+ 5631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:59103: BlockingRecv error: recv got EOF from 127.23.200.194:59103 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 07 20:13:36 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:36 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.195@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:36.940514 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:36.936141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.195:43649)
0807 20:13:36.936281 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:36.936294 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:36.936359 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:36.936485 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:36.936486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:36.936656 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:36.936703 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:36.936704 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:36.937388 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:36.937389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:36.937921 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:36.937923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:36.938256 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:36.938257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:36.938292 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:36.938721 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:36.938725 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:36.940453 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
W20260807 20:13:36.940621 7063 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:36.936231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:43649 (local address 127.23.200.254:42395)
0807 20:13:36.936366 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:36.936367 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:36.936377 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:13:36.936417 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:36.936418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:36.936431 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:36.936473 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:36.936792 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:36.937307 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:36.938003 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:36.938199 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:36.938258 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:36.940552 (+ 2294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.195:43649: BlockingRecv error: recv got EOF from 127.23.200.195:43649 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 07 20:13:37 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:37 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.193@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:37.695660 7070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:37.691152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.193:59427)
0807 20:13:37.691321 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:37.691335 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:37.691403 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:37.691524 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:37.691525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:37.691696 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:37.691747 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:37.691749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:37.692404 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:37.692405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:37.692965 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:37.692967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:37.693313 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:37.693314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:37.693361 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:37.693833 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:37.693838 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:37.695585 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":94,"threads_started":1}
W20260807 20:13:37.695765 7071 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:37.691255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:59427 (local address 127.23.200.254:42395)
0807 20:13:37.691410 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:37.691412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:37.691422 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:13:37.691445 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:37.691445 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:37.691456 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:37.691515 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:37.691804 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:37.692318 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:37.693038 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:37.693255 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:37.693303 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:37.695693 (+ 2390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.193:59427: BlockingRecv error: recv got EOF from 127.23.200.193:59427 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
Aug 07 20:13:37 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:13:37 dist-test-slave-2j7r krb5kdc[5217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.200.194@KRBTEST.COM for oryx/127.23.200.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:13:37.829276 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:37.824294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:42395 (local address 127.23.200.194:42501)
0807 20:13:37.824452 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:37.824463 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:37.824526 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:37.824648 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:37.824649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:37.824809 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:37.824859 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:37.824860 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:37.825642 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:37.825644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:37.826456 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:37.826460 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:37.826788 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:37.826789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:37.826830 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:37.827304 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:37.827309 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:37.829212 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.200.254:42395: Server oryx/127.23.200.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":64,"threads_started":1}
W20260807 20:13:37.829463 7071 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:37.824392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:42501 (local address 127.23.200.254:42395)
0807 20:13:37.824445 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:37.824447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:37.824510 (+ 63us) server_negotiation.cc:407] Connection header received
0807 20:13:37.824585 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:37.824586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:37.824598 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:37.824635 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:37.824952 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:37.825570 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:37.826532 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:37.826725 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:37.826792 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:37.829403 (+ 2611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.200.194:42501: BlockingRecv error: recv got EOF from 127.23.200.194:42501 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:13:37.868602 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 6253
I20260807 20:13:37.882154 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 6530
I20260807 20:13:37.896158 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 6753
I20260807 20:13:37.904835 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 6123
2026-08-07T20:13:37Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17642 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T20:13:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:37Z Disabled control of system clock
I20260807 20:13:37.938407 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:45221
--webserver_interface=127.23.200.254
--webserver_port=0
--builtin_ntp_servers=127.23.200.212:37687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:45221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 20:13:38.020675 7076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:38.020886 7076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:38.020905 7076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:38.022172 7076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:38.022222 7076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:38.022234 7076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:38.022245 7076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:38.022256 7076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:13:38.023679 7076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:37687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:45221
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:45221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:38.023928 7076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:38.024224 7076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:38.029157 7082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:38.029163 7081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:38.029178 7085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:38.029327 7076 server_base.cc:1034] running on GCE node
I20260807 20:13:38.029672 7076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:38.029987 7076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:38.031131 7076 hybrid_clock.cc:648] HybridClock initialized: now 1786133618031119 us; error 32 us; skew 500 ppm
I20260807 20:13:38.032514 7076 webserver.cc:460] Webserver started at http://127.23.200.254:44269/ using document root <none> and password file <none>
I20260807 20:13:38.032773 7076 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:38.032824 7076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:38.032956 7076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:38.034124 7076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "fde2a834055a40b9a406c9b48e9f9114"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "434ea28b523528b286b409642eaefaa3"
server_key_iv: "e0d8c62bae94580b27018d11e2022b71"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.034497 7076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "fde2a834055a40b9a406c9b48e9f9114"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "434ea28b523528b286b409642eaefaa3"
server_key_iv: "e0d8c62bae94580b27018d11e2022b71"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.035809 7076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:13:38.036592 7096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.036767 7076 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:38.036857 7076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "fde2a834055a40b9a406c9b48e9f9114"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "434ea28b523528b286b409642eaefaa3"
server_key_iv: "e0d8c62bae94580b27018d11e2022b71"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.036934 7076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:38.058084 7076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:38.058427 7076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:38.058566 7076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:38.062585 7076 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:45221
I20260807 20:13:38.062645 7196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:45221 every 8 connection(s)
I20260807 20:13:38.062988 7076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:13:38.063679 7197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:38.066408 7197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114: Bootstrap starting.
I20260807 20:13:38.067025 7197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:38.067281 7197 log.cc:824] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:38.067950 7197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114: No bootstrap required, opened a new log
I20260807 20:13:38.069541 7197 raft_consensus.cc:348] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde2a834055a40b9a406c9b48e9f9114" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45221 } }
I20260807 20:13:38.069682 7197 raft_consensus.cc:374] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:38.069703 7197 raft_consensus.cc:729] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fde2a834055a40b9a406c9b48e9f9114, State: Initialized, Role: FOLLOWER
I20260807 20:13:38.069819 7197 consensus_queue.cc:260] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde2a834055a40b9a406c9b48e9f9114" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45221 } }
I20260807 20:13:38.069893 7197 raft_consensus.cc:388] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:13:38.069916 7197 raft_consensus.cc:482] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:13:38.069948 7197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:38.070470 7197 raft_consensus.cc:504] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde2a834055a40b9a406c9b48e9f9114" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45221 } }
I20260807 20:13:38.070559 7197 leader_election.cc:302] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fde2a834055a40b9a406c9b48e9f9114; no voters:
I20260807 20:13:38.070700 7197 leader_election.cc:288] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:13:38.070768 7206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:38.070962 7197 sys_catalog.cc:564] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:38.070993 7206 raft_consensus.cc:686] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [term 1 LEADER]: Becoming Leader. State: Replica: fde2a834055a40b9a406c9b48e9f9114, State: Running, Role: LEADER
I20260807 20:13:38.071172 7206 consensus_queue.cc:237] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde2a834055a40b9a406c9b48e9f9114" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45221 } }
I20260807 20:13:38.071518 7209 sys_catalog.cc:455] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fde2a834055a40b9a406c9b48e9f9114. Latest consensus state: current_term: 1 leader_uuid: "fde2a834055a40b9a406c9b48e9f9114" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde2a834055a40b9a406c9b48e9f9114" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45221 } } }
I20260807 20:13:38.071667 7207 sys_catalog.cc:455] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fde2a834055a40b9a406c9b48e9f9114" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde2a834055a40b9a406c9b48e9f9114" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45221 } } }
I20260807 20:13:38.071676 7209 sys_catalog.cc:458] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:38.071766 7207 sys_catalog.cc:458] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:38.072214 7221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:13:38.072732 7221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:38.073065 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 7076
I20260807 20:13:38.073179 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:13:38.073380 24355 external_mini_cluster.cc:1428] Setting key 696488a1781f0298ac9e234e0484d089
I20260807 20:13:38.074625 7221 catalog_manager.cc:1347] Generated new cluster ID: 2b6e0e51fc9b4ff28782e9039b384d8e
I20260807 20:13:38.074678 7221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:13:38.081421 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:38.073889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38166 (local address 127.23.200.254:45221)
0807 20:13:38.077257 (+ 3368us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:38.077266 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:38.077307 (+ 41us) server_negotiation.cc:407] Connection header received
0807 20:13:38.077421 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:38.077427 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:38.077483 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:38.077623 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:38.077876 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:38.078793 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:38.079553 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:38.080003 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:38.080117 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:38.080762 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:38.080775 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:38.080775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:38.080946 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:38.080981 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:38.080993 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:38.081047 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:38.081166 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 20:13:38.081252 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3291,"thread_start_us":68,"threads_started":1}
I20260807 20:13:38.085462 7221 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:13:38.085968 7221 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:38.093158 7221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fde2a834055a40b9a406c9b48e9f9114: Generated new TSK 0
I20260807 20:13:38.093329 7221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:13:38.100549 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45221
--builtin_ntp_servers=127.23.200.212:37687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 20:13:38.197103 7239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:38.197319 7239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:38.197340 7239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:38.198500 7239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:38.198540 7239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:38.198565 7239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:13:38.199934 7239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:37687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:38.200165 7239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:38.200454 7239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:38.206421 7246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:38.206482 7251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:38.206548 7239 server_base.cc:1034] running on GCE node
W20260807 20:13:38.206727 7248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:38.206954 7239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:38.207217 7239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:38.208400 7239 hybrid_clock.cc:648] HybridClock initialized: now 1786133618208374 us; error 41 us; skew 500 ppm
I20260807 20:13:38.209753 7239 webserver.cc:460] Webserver started at http://127.23.200.193:34605/ using document root <none> and password file <none>
I20260807 20:13:38.209980 7239 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:38.210031 7239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:38.210165 7239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:38.211231 7239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "8a4c7038ffbc4c65b1d0201ab8309e6e"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "0fa21606db18887a8362c10991f6e669"
server_key_iv: "2597350d2c9aeded0bf97bcbe032879a"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.211577 7239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "8a4c7038ffbc4c65b1d0201ab8309e6e"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "0fa21606db18887a8362c10991f6e669"
server_key_iv: "2597350d2c9aeded0bf97bcbe032879a"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.212872 7239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:13:38.213706 7262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.213910 7239 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:13:38.214006 7239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "8a4c7038ffbc4c65b1d0201ab8309e6e"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "0fa21606db18887a8362c10991f6e669"
server_key_iv: "2597350d2c9aeded0bf97bcbe032879a"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.214082 7239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:38.235877 7239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:38.236214 7239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:38.236348 7239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:38.236596 7239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:38.236934 7239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:38.236971 7239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.237001 7239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:38.237025 7239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.242982 7239 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:45165
I20260807 20:13:38.243032 7449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:45165 every 8 connection(s)
I20260807 20:13:38.243371 7239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:13:38.247073 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 7239
I20260807 20:13:38.247166 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:13:38.247373 24355 external_mini_cluster.cc:1428] Setting key 25883c2cf132a250a948eb23bbdccc43
I20260807 20:13:38.248406 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45221
--builtin_ntp_servers=127.23.200.212:37687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 20:13:38.248659 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:38.244079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:34031 (local address 127.23.200.254:45221)
0807 20:13:38.244144 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:38.244147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:38.244382 (+ 235us) server_negotiation.cc:407] Connection header received
0807 20:13:38.244881 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:38.244882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:38.244895 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:38.244955 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:38.245642 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:38.246157 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:38.247129 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:38.247413 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:38.247486 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:38.248378 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:38.248380 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:38.248380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:38.248385 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:38.248404 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:38.248408 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:38.248463 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:38.248547 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 20:13:38.248584 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:13:38.248883 7452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:38.244068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45221 (local address 127.23.200.193:34031)
0807 20:13:38.244264 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:38.244299 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:38.244788 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:38.245005 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:38.245009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:38.245017 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:38.245568 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:38.245573 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:38.246227 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:38.246227 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:38.247052 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:38.247055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:38.247523 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:38.247524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:38.247690 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:38.248234 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:13:38.248235 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:38.248265 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:13:38.248288 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:38.248475 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:38.248479 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:38.248481 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:13:38.248539 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 20:13:38.248652 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
I20260807 20:13:38.249445 7450 heartbeater.cc:347] Connected to a master server at 127.23.200.254:45221
I20260807 20:13:38.249531 7450 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:38.249708 7450 heartbeater.cc:511] Master 127.23.200.254:45221 requested a full tablet report, sending...
I20260807 20:13:38.250262 7123 ts_manager.cc:194] Registered new tserver with Master: 8a4c7038ffbc4c65b1d0201ab8309e6e (127.23.200.193:45165)
I20260807 20:13:38.250854 7123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.200.193:34031
W20260807 20:13:38.332804 7453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:38.333037 7453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:38.333076 7453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:38.334267 7453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:38.334311 7453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:38.334345 7453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:13:38.335692 7453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:37687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:38.335934 7453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:38.336236 7453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:38.341434 7464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:38.341491 7460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:38.341445 7459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:38.341606 7453 server_base.cc:1034] running on GCE node
I20260807 20:13:38.341900 7453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:38.342140 7453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:38.343312 7453 hybrid_clock.cc:648] HybridClock initialized: now 1786133618343330 us; error 61 us; skew 500 ppm
I20260807 20:13:38.344544 7453 webserver.cc:460] Webserver started at http://127.23.200.194:34707/ using document root <none> and password file <none>
I20260807 20:13:38.344762 7453 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:38.344811 7453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:38.344941 7453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:38.346135 7453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "e5191b2f38f84b68835fe6a587b941d6"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "6df73fd4a43d7b0500e4047e1279c55e"
server_key_iv: "9f2c9aea45460d1fb20878fb2905a557"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.346475 7453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "e5191b2f38f84b68835fe6a587b941d6"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "6df73fd4a43d7b0500e4047e1279c55e"
server_key_iv: "9f2c9aea45460d1fb20878fb2905a557"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.347738 7453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:13:38.348454 7473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.348639 7453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:38.348721 7453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "e5191b2f38f84b68835fe6a587b941d6"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "6df73fd4a43d7b0500e4047e1279c55e"
server_key_iv: "9f2c9aea45460d1fb20878fb2905a557"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.348788 7453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:38.354910 7453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:38.355155 7453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:38.355284 7453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:38.355477 7453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:38.355767 7453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:38.355799 7453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.355825 7453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:38.355849 7453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.361347 7453 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:42265
I20260807 20:13:38.361402 7669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:42265 every 8 connection(s)
I20260807 20:13:38.361703 7453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:13:38.364347 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 7453
I20260807 20:13:38.364468 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:13:38.364645 24355 external_mini_cluster.cc:1428] Setting key 47dd15fe8e17512f2ace2e543853ef74
I20260807 20:13:38.365584 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45221
--builtin_ntp_servers=127.23.200.212:37687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 20:13:38.366904 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:38.362398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:42257 (local address 127.23.200.254:45221)
0807 20:13:38.362453 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:38.362455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:38.362672 (+ 217us) server_negotiation.cc:407] Connection header received
0807 20:13:38.363170 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:38.363171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:38.363182 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:38.363249 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:38.363940 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:38.364476 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:38.365550 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:38.365804 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:38.365862 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:38.366641 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:38.366643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:38.366643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:38.366649 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:38.366664 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:38.366669 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:38.366716 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:38.366792 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 20:13:38.366822 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 20:13:38.367095 7676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:38.362361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45221 (local address 127.23.200.194:42257)
0807 20:13:38.362555 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:38.362595 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:38.363051 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:38.363316 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:38.363320 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:38.363328 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:38.363847 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:38.363851 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:38.364562 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:38.364563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:38.365467 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:38.365470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:38.365879 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:38.365881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:38.366019 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:38.366510 (+ 491us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:13:38.366510 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:38.366537 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:13:38.366561 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:38.366721 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:38.366727 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:38.366731 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:13:38.366790 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 20:13:38.366892 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260807 20:13:38.367630 7672 heartbeater.cc:347] Connected to a master server at 127.23.200.254:45221
I20260807 20:13:38.367723 7672 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:38.367904 7672 heartbeater.cc:511] Master 127.23.200.254:45221 requested a full tablet report, sending...
I20260807 20:13:38.368275 7123 ts_manager.cc:194] Registered new tserver with Master: e5191b2f38f84b68835fe6a587b941d6 (127.23.200.194:42265)
I20260807 20:13:38.368628 7123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.200.194:42257
W20260807 20:13:38.455068 7677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:38.455358 7677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:38.455396 7677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:38.456707 7677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:38.456775 7677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:38.456818 7677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:13:38.458384 7677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:37687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:38.458634 7677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:38.458945 7677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:38.464958 7691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:38.464967 7690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:38.465127 7677 server_base.cc:1034] running on GCE node
W20260807 20:13:38.465252 7696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:38.465448 7677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:38.465667 7677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:38.466805 7677 hybrid_clock.cc:648] HybridClock initialized: now 1786133618466792 us; error 30 us; skew 500 ppm
I20260807 20:13:38.468056 7677 webserver.cc:460] Webserver started at http://127.23.200.195:34021/ using document root <none> and password file <none>
I20260807 20:13:38.468266 7677 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:38.468314 7677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:38.468505 7677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:38.469790 7677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "7b46f7473b474a7295d46d17c16228b5"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "f91e844c503e8a489ac15f8b7a81d98a"
server_key_iv: "9c5eb060f425e5645ab88cb5ebcd89d1"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.470229 7677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "7b46f7473b474a7295d46d17c16228b5"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "f91e844c503e8a489ac15f8b7a81d98a"
server_key_iv: "9c5eb060f425e5645ab88cb5ebcd89d1"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.471822 7677 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:38.472695 7704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.472870 7677 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:38.473001 7677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "7b46f7473b474a7295d46d17c16228b5"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "f91e844c503e8a489ac15f8b7a81d98a"
server_key_iv: "9c5eb060f425e5645ab88cb5ebcd89d1"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.473155 7677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:38.492738 7677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:38.493054 7677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:38.493227 7677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:38.493474 7677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:38.493840 7677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:38.493886 7677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.493923 7677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:38.493947 7677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.499583 7677 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:39149
I20260807 20:13:38.499624 7884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:39149 every 8 connection(s)
I20260807 20:13:38.499964 7677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:13:38.501523 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 7677
I20260807 20:13:38.501605 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:13:38.501820 24355 external_mini_cluster.cc:1428] Setting key d334ae667a14a062b0eb75a150abf3a0
I20260807 20:13:38.505383 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:38.500707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:60393 (local address 127.23.200.254:45221)
0807 20:13:38.500772 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:38.500775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:38.501001 (+ 226us) server_negotiation.cc:407] Connection header received
0807 20:13:38.501637 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:38.501639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:38.501652 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:38.501721 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:38.502529 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:38.503091 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:38.504057 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:38.504244 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:38.504296 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:38.505167 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:38.505168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:38.505168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:38.505173 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:38.505190 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:38.505194 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:38.505241 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:38.505298 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 20:13:38.505323 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 20:13:38.505594 7888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:38.500680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45221 (local address 127.23.200.195:60393)
0807 20:13:38.500881 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:38.500919 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:38.501521 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:38.501787 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:38.501791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:38.501800 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:38.502459 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:38.502466 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:38.503170 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:38.503171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:38.503979 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:38.503982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:38.504355 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:38.504357 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:38.504479 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:38.505013 (+ 534us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:13:38.505014 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:38.505049 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:13:38.505072 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:38.505248 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:38.505251 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:38.505254 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:13:38.505301 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 20:13:38.505382 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260807 20:13:38.506038 7886 heartbeater.cc:347] Connected to a master server at 127.23.200.254:45221
I20260807 20:13:38.506125 7886 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:38.506276 7886 heartbeater.cc:511] Master 127.23.200.254:45221 requested a full tablet report, sending...
I20260807 20:13:38.506618 7123 ts_manager.cc:194] Registered new tserver with Master: 7b46f7473b474a7295d46d17c16228b5 (127.23.200.195:39149)
I20260807 20:13:38.506951 7123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.200.195:60393
I20260807 20:13:38.513536 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 20:13:38.515345 7236 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:38.514855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38170 (local address 127.23.200.254:45221)
0807 20:13:38.514908 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:38.514911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:38.515011 (+ 100us) server_negotiation.cc:407] Connection header received
0807 20:13:38.515078 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:38.515079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:38.515091 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:38.515143 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:38.515279 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38170: BlockingRecv error: recv got EOF from 127.0.0.1:38170 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 20:13:38.515432 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 7239
I20260807 20:13:38.533736 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 7453
I20260807 20:13:38.542183 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 7677
I20260807 20:13:38.554476 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 7076
2026-08-07T20:13:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (657 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T20:13:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:38Z Disabled control of system clock
I20260807 20:13:38.596060 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:37303
--webserver_interface=127.23.200.254
--webserver_port=0
--builtin_ntp_servers=127.23.200.212:41133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:37303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 20:13:38.679145 7902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:38.679376 7902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:38.679397 7902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:38.680603 7902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:38.680647 7902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:38.680658 7902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:38.680670 7902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:38.680680 7902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:13:38.682108 7902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:41133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:37303
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:37303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:38.682372 7902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:38.682677 7902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:13:38.687788 7902 server_base.cc:1034] running on GCE node
W20260807 20:13:38.687740 7908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:38.687767 7910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:38.687747 7914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:38.688242 7902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:38.688570 7902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:38.689721 7902 hybrid_clock.cc:648] HybridClock initialized: now 1786133618689704 us; error 33 us; skew 500 ppm
I20260807 20:13:38.691013 7902 webserver.cc:460] Webserver started at http://127.23.200.254:41283/ using document root <none> and password file <none>
I20260807 20:13:38.691227 7902 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:38.691337 7902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:38.691469 7902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:38.692556 7902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "a04e4b484b4a4d7c9e1091877912c19e"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "cbcd338b1551eef2e20954085ddff10a"
server_key_iv: "01e376f9291a96fd8267492ecd43b5dd"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.692880 7902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "a04e4b484b4a4d7c9e1091877912c19e"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "cbcd338b1551eef2e20954085ddff10a"
server_key_iv: "01e376f9291a96fd8267492ecd43b5dd"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.694193 7902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:13:38.695036 7926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.695221 7902 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:38.695318 7902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "a04e4b484b4a4d7c9e1091877912c19e"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "cbcd338b1551eef2e20954085ddff10a"
server_key_iv: "01e376f9291a96fd8267492ecd43b5dd"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.695389 7902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:38.698809 7902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:38.699080 7902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:38.699200 7902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:38.703020 7902 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:37303
I20260807 20:13:38.703052 8012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:37303 every 8 connection(s)
I20260807 20:13:38.703369 7902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:13:38.704257 8015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:38.706517 8015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e: Bootstrap starting.
I20260807 20:13:38.707051 8015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:38.707288 8015 log.cc:824] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:38.707895 8015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e: No bootstrap required, opened a new log
I20260807 20:13:38.709482 8015 raft_consensus.cc:348] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04e4b484b4a4d7c9e1091877912c19e" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37303 } }
I20260807 20:13:38.709615 8015 raft_consensus.cc:374] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:38.709636 8015 raft_consensus.cc:729] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a04e4b484b4a4d7c9e1091877912c19e, State: Initialized, Role: FOLLOWER
I20260807 20:13:38.709738 8015 consensus_queue.cc:260] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04e4b484b4a4d7c9e1091877912c19e" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37303 } }
I20260807 20:13:38.709794 8015 raft_consensus.cc:388] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:13:38.709822 8015 raft_consensus.cc:482] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:13:38.709861 8015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:38.710361 8015 raft_consensus.cc:504] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04e4b484b4a4d7c9e1091877912c19e" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37303 } }
I20260807 20:13:38.710456 8015 leader_election.cc:302] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a04e4b484b4a4d7c9e1091877912c19e; no voters:
I20260807 20:13:38.710618 8015 leader_election.cc:288] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:13:38.710700 8019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:38.710807 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 7902
I20260807 20:13:38.710865 8019 raft_consensus.cc:686] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [term 1 LEADER]: Becoming Leader. State: Replica: a04e4b484b4a4d7c9e1091877912c19e, State: Running, Role: LEADER
I20260807 20:13:38.710906 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:13:38.710947 8015 sys_catalog.cc:564] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:38.710952 8019 consensus_queue.cc:237] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04e4b484b4a4d7c9e1091877912c19e" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37303 } }
I20260807 20:13:38.711100 24355 external_mini_cluster.cc:1428] Setting key e1e719a13f7bc4d8c8237e2277f5db20
I20260807 20:13:38.711478 8020 sys_catalog.cc:455] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a04e4b484b4a4d7c9e1091877912c19e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04e4b484b4a4d7c9e1091877912c19e" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37303 } } }
I20260807 20:13:38.711558 8020 sys_catalog.cc:458] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:38.712208 8022 sys_catalog.cc:455] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a04e4b484b4a4d7c9e1091877912c19e. Latest consensus state: current_term: 1 leader_uuid: "a04e4b484b4a4d7c9e1091877912c19e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a04e4b484b4a4d7c9e1091877912c19e" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37303 } } }
I20260807 20:13:38.712476 8040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:13:38.712297 8022 sys_catalog.cc:458] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:38.713002 8040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:38.713866 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:38.711798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48212 (local address 127.23.200.254:37303)
0807 20:13:38.712050 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:38.712060 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:38.712324 (+ 264us) server_negotiation.cc:407] Connection header received
0807 20:13:38.712449 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:38.712456 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:38.712503 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:38.712636 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:38.713363 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:38.713367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:38.713367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:38.713371 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:38.713406 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:38.713420 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:38.713493 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:38.713601 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 20:13:38.713665 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":70,"threads_started":1}
I20260807 20:13:38.714664 8040 catalog_manager.cc:1347] Generated new cluster ID: 3ed030068a4a4b6aa730550b50d4aa19
I20260807 20:13:38.714715 8040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:13:38.728835 8040 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:13:38.729010 8040 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:38.735816 8040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a04e4b484b4a4d7c9e1091877912c19e: Generated new TSK 0
I20260807 20:13:38.736044 8040 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:13:38.741564 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37303
--builtin_ntp_servers=127.23.200.212:41133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 20:13:38.825518 8058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:38.825829 8058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:38.825874 8058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:38.827251 8058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:38.827293 8058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:38.827318 8058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:13:38.828665 8058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:41133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:38.828881 8058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:38.829207 8058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:38.834424 8065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:38.834460 8064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:38.834419 8067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:38.834904 8058 server_base.cc:1034] running on GCE node
I20260807 20:13:38.835122 8058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:38.835378 8058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:38.836556 8058 hybrid_clock.cc:648] HybridClock initialized: now 1786133618836539 us; error 33 us; skew 500 ppm
I20260807 20:13:38.837867 8058 webserver.cc:460] Webserver started at http://127.23.200.193:44313/ using document root <none> and password file <none>
I20260807 20:13:38.838073 8058 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:38.838114 8058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:38.838207 8058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:38.839228 8058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "8cfd59bac6b940c6992835c696bb9741"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "1db8c5237c4ad2c838de46f5d70093a3"
server_key_iv: "853cf202e1dfd6d261687966441d0b1c"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.839567 8058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "8cfd59bac6b940c6992835c696bb9741"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "1db8c5237c4ad2c838de46f5d70093a3"
server_key_iv: "853cf202e1dfd6d261687966441d0b1c"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.840855 8058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:13:38.841715 8074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.841912 8058 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:13:38.841998 8058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "8cfd59bac6b940c6992835c696bb9741"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "1db8c5237c4ad2c838de46f5d70093a3"
server_key_iv: "853cf202e1dfd6d261687966441d0b1c"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.842056 8058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:38.845329 8058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:38.845566 8058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:38.845659 8058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:38.845821 8058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:38.846093 8058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:38.846122 8058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.846141 8058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:38.846158 8058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.851570 8058 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:35191
I20260807 20:13:38.851698 8258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:35191 every 8 connection(s)
I20260807 20:13:38.852043 8058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:13:38.855280 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:38.852837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:48639 (local address 127.23.200.254:37303)
0807 20:13:38.852894 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:38.852896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:38.853265 (+ 369us) server_negotiation.cc:407] Connection header received
0807 20:13:38.853769 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:38.853770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:38.853780 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:38.853837 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:38.855023 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:38.855024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:38.855024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:38.855029 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:38.855049 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:38.855054 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:38.855121 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:38.855180 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 20:13:38.855194 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:13:38.855399 8265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:38.852804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37303 (local address 127.23.200.193:48639)
0807 20:13:38.853067 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:38.853142 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:38.853654 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:38.853893 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:38.853896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:38.853905 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:38.854838 (+ 933us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:13:38.854839 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:38.854869 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:13:38.854905 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:38.855120 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:38.855123 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:38.855126 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:13:38.855175 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 20:13:38.855225 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":170,"threads_started":1}
I20260807 20:13:38.855942 8259 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37303
I20260807 20:13:38.856047 8259 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:38.856200 8259 heartbeater.cc:511] Master 127.23.200.254:37303 requested a full tablet report, sending...
I20260807 20:13:38.856750 7960 ts_manager.cc:194] Registered new tserver with Master: 8cfd59bac6b940c6992835c696bb9741 (127.23.200.193:35191)
I20260807 20:13:38.857028 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 8058
I20260807 20:13:38.857141 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:13:38.857393 24355 external_mini_cluster.cc:1428] Setting key 3792ef095660f8e212f46cdffd2ab989
I20260807 20:13:38.858362 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37303
--builtin_ntp_servers=127.23.200.212:41133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 20:13:38.945621 8266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:38.945842 8266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:38.945873 8266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:38.947098 8266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:38.947137 8266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:38.947171 8266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:13:38.948563 8266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:41133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:38.948800 8266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:38.949077 8266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:38.954160 8273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:38.954288 8271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:38.954161 8275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:38.954181 8266 server_base.cc:1034] running on GCE node
I20260807 20:13:38.954612 8266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:38.954854 8266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:38.956018 8266 hybrid_clock.cc:648] HybridClock initialized: now 1786133618956004 us; error 28 us; skew 500 ppm
I20260807 20:13:38.957378 8266 webserver.cc:460] Webserver started at http://127.23.200.194:42995/ using document root <none> and password file <none>
I20260807 20:13:38.957603 8266 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:38.957652 8266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:38.957780 8266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:38.958843 8266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "8bd9a8698d00498abb7e3fe7e16cfd9a"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "30a94f3960ea9785f341aac20786cadf"
server_key_iv: "b29235fa75958d7b6a8db0acf4889d99"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.959177 8266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "8bd9a8698d00498abb7e3fe7e16cfd9a"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "30a94f3960ea9785f341aac20786cadf"
server_key_iv: "b29235fa75958d7b6a8db0acf4889d99"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.960526 8266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:13:38.961364 8285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.961524 8266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:13:38.961611 8266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "8bd9a8698d00498abb7e3fe7e16cfd9a"
format_stamp: "Formatted at 2026-08-07 20:13:38 on dist-test-slave-2j7r"
server_key: "30a94f3960ea9785f341aac20786cadf"
server_key_iv: "b29235fa75958d7b6a8db0acf4889d99"
server_key_version: "encryptionkey@0"
I20260807 20:13:38.961675 8266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:38.964994 8266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:38.965277 8266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:38.965411 8266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:38.965606 8266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:38.965888 8266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:38.965924 8266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.965950 8266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:38.965986 8266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:38.971622 8266 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:45789
I20260807 20:13:38.971678 8475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:45789 every 8 connection(s)
I20260807 20:13:38.971976 8266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:13:38.974992 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:38.972798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:46253 (local address 127.23.200.254:37303)
0807 20:13:38.972872 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:38.972874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:38.973076 (+ 202us) server_negotiation.cc:407] Connection header received
0807 20:13:38.973644 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:38.973645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:38.973650 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:38.973699 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:38.974807 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:38.974807 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:38.974807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:38.974809 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:38.974820 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:38.974823 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:38.974862 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:38.974928 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 20:13:38.974938 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:13:38.975159 8482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:38.972753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37303 (local address 127.23.200.194:46253)
0807 20:13:38.972957 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:38.972997 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:38.973544 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:38.973780 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:38.973784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:38.973794 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:38.974683 (+ 889us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:13:38.974683 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:38.974712 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:13:38.974739 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:38.974866 (+ 127us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:38.974869 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:38.974871 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:13:38.974919 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 20:13:38.974966 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260807 20:13:38.975658 8479 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37303
I20260807 20:13:38.975780 8479 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:38.975943 8479 heartbeater.cc:511] Master 127.23.200.254:37303 requested a full tablet report, sending...
I20260807 20:13:38.976332 7960 ts_manager.cc:194] Registered new tserver with Master: 8bd9a8698d00498abb7e3fe7e16cfd9a (127.23.200.194:45789)
I20260807 20:13:38.977242 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 8266
I20260807 20:13:38.977337 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:13:38.977537 24355 external_mini_cluster.cc:1428] Setting key 1a8365134ac0bdafd96b80e82dace0f5
I20260807 20:13:38.978456 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37303
--builtin_ntp_servers=127.23.200.212:41133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 20:13:39.061316 8483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:39.061527 8483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:39.061558 8483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:39.062708 8483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:39.062749 8483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:39.062783 8483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:13:39.064150 8483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:41133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:39.064397 8483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:39.064678 8483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:39.069751 8494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:39.069761 8491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:39.069775 8489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:39.070009 8483 server_base.cc:1034] running on GCE node
I20260807 20:13:39.070185 8483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:39.070401 8483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:39.071552 8483 hybrid_clock.cc:648] HybridClock initialized: now 1786133619071542 us; error 34 us; skew 500 ppm
I20260807 20:13:39.072781 8483 webserver.cc:460] Webserver started at http://127.23.200.195:45151/ using document root <none> and password file <none>
I20260807 20:13:39.073045 8483 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:39.073124 8483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:39.073257 8483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:39.074280 8483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "5012c63540e74b9e93e02ebd3a685c23"
format_stamp: "Formatted at 2026-08-07 20:13:39 on dist-test-slave-2j7r"
server_key: "c36ef44ee007baf16b67742ae1a02a66"
server_key_iv: "fc0596f7a7c69f7a09452630d9dce7f9"
server_key_version: "encryptionkey@0"
I20260807 20:13:39.074605 8483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "5012c63540e74b9e93e02ebd3a685c23"
format_stamp: "Formatted at 2026-08-07 20:13:39 on dist-test-slave-2j7r"
server_key: "c36ef44ee007baf16b67742ae1a02a66"
server_key_iv: "fc0596f7a7c69f7a09452630d9dce7f9"
server_key_version: "encryptionkey@0"
I20260807 20:13:39.075845 8483 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:13:39.076551 8505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:39.076710 8483 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:39.076807 8483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "5012c63540e74b9e93e02ebd3a685c23"
format_stamp: "Formatted at 2026-08-07 20:13:39 on dist-test-slave-2j7r"
server_key: "c36ef44ee007baf16b67742ae1a02a66"
server_key_iv: "fc0596f7a7c69f7a09452630d9dce7f9"
server_key_version: "encryptionkey@0"
I20260807 20:13:39.076869 8483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:39.080202 8483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:39.080459 8483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:39.080580 8483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:39.080777 8483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:39.081120 8483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:39.081163 8483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:39.081195 8483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:39.081219 8483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:39.086880 8483 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:43407
I20260807 20:13:39.086931 8683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:43407 every 8 connection(s)
I20260807 20:13:39.087244 8483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:13:39.090210 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:39.087986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:50197 (local address 127.23.200.254:37303)
0807 20:13:39.088061 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:39.088064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:39.088227 (+ 163us) server_negotiation.cc:407] Connection header received
0807 20:13:39.088760 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:39.088761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:39.088769 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:39.088841 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:39.090015 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:39.090017 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:39.090017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:13:39.090022 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:39.090037 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:13:39.090041 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:39.090091 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:39.090143 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 20:13:39.090155 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 20:13:39.090369 8689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:39.087913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37303 (local address 127.23.200.195:50197)
0807 20:13:39.088105 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:39.088138 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:39.088615 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:39.088896 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:39.088907 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:39.088915 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:39.089850 (+ 935us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:13:39.089851 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:39.089880 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:13:39.089925 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:39.090088 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:39.090091 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:39.090093 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:13:39.090141 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 20:13:39.090189 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260807 20:13:39.090821 8685 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37303
I20260807 20:13:39.090903 8685 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:39.091023 8685 heartbeater.cc:511] Master 127.23.200.254:37303 requested a full tablet report, sending...
I20260807 20:13:39.091343 7960 ts_manager.cc:194] Registered new tserver with Master: 5012c63540e74b9e93e02ebd3a685c23 (127.23.200.195:43407)
I20260807 20:13:39.093840 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 8483
I20260807 20:13:39.093927 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:13:39.094102 24355 external_mini_cluster.cc:1428] Setting key e944de64ca2d90db414d5e00cb8a004c
I20260807 20:13:39.094926 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 20:13:39.096648 8038 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:13:39.096192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48220 (local address 127.23.200.254:37303)
0807 20:13:39.096316 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:39.096318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:39.096358 (+ 40us) server_negotiation.cc:407] Connection header received
0807 20:13:39.096427 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:39.096428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:39.096432 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:39.096475 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:39.096588 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48220: BlockingRecv error: recv got EOF from 127.0.0.1:48220 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 20:13:39.096760 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 8058
I20260807 20:13:39.113801 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 8266
I20260807 20:13:39.128293 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 8483
I20260807 20:13:39.161968 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 7902
2026-08-07T20:13:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (601 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:13:39 dist-test-slave-2j7r krb5kdc[8699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:13:39 dist-test-slave-2j7r krb5kdc[8699](info): set up 2 sockets
Aug 07 20:13:39 dist-test-slave-2j7r krb5kdc[8699](info): commencing operation
krb5kdc: starting...
W20260807 20:13:40.317677 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:13:40 dist-test-slave-2j7r krb5kdc[8699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:13:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:13:40.439678 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:41671
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:38519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:41671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:40.523506 8724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:40.523712 8724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:40.523741 8724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:40.524873 8724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:13:40.524912 8724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:40.524931 8724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:40.524946 8724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:13:40.524963 8724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:13:40.526527 8724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:38519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:41671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:41671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:40.526782 8724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:40.526999 8724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:40.531786 8733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:40.531813 8734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:40.531780 8737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:40.532100 8724 server_base.cc:1034] running on GCE node
I20260807 20:13:40.532335 8724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:40.532608 8724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:40.533749 8724 hybrid_clock.cc:648] HybridClock initialized: now 1786133620533734 us; error 28 us; skew 500 ppm
Aug 07 20:13:40 dist-test-slave-2j7r krb5kdc[8699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133620, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:40.536100 8724 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:13:40.536485 8724 webserver.cc:460] Webserver started at http://127.23.200.254:43643/ using document root <none> and password file <none>
I20260807 20:13:40.536666 8724 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:40.536705 8724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:40.536791 8724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:40.537801 8724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "ebda05f9d98d45d7a7c3a4a6ae268ae6"
format_stamp: "Formatted at 2026-08-07 20:13:40 on dist-test-slave-2j7r"
server_key: "5a33c53b9a774e4813e3ba01f4a8960f"
server_key_iv: "6b5e8f611891ea588b91f1387b45f920"
server_key_version: "encryptionkey@0"
I20260807 20:13:40.538084 8724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "ebda05f9d98d45d7a7c3a4a6ae268ae6"
format_stamp: "Formatted at 2026-08-07 20:13:40 on dist-test-slave-2j7r"
server_key: "5a33c53b9a774e4813e3ba01f4a8960f"
server_key_iv: "6b5e8f611891ea588b91f1387b45f920"
server_key_version: "encryptionkey@0"
I20260807 20:13:40.539273 8724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:13:40.539928 8747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:40.540093 8724 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:40.540174 8724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "ebda05f9d98d45d7a7c3a4a6ae268ae6"
format_stamp: "Formatted at 2026-08-07 20:13:40 on dist-test-slave-2j7r"
server_key: "5a33c53b9a774e4813e3ba01f4a8960f"
server_key_iv: "6b5e8f611891ea588b91f1387b45f920"
server_key_version: "encryptionkey@0"
I20260807 20:13:40.540228 8724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:40.562295 8724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:40.564471 8724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:40.564597 8724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:40.568521 8724 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:41671
I20260807 20:13:40.568557 8841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:41671 every 8 connection(s)
I20260807 20:13:40.568877 8724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:13:40.569900 8842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:40.572297 8842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6: Bootstrap starting.
I20260807 20:13:40.572875 8842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:40.573199 8842 log.cc:824] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:40.573922 8842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6: No bootstrap required, opened a new log
I20260807 20:13:40.575109 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 8724
I20260807 20:13:40.575212 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:13:40.575403 24355 external_mini_cluster.cc:1428] Setting key 7019ef11b05d646239c9902bde82bc25
I20260807 20:13:40.575826 8842 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebda05f9d98d45d7a7c3a4a6ae268ae6" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41671 } }
I20260807 20:13:40.576016 8842 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:40.576090 8842 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebda05f9d98d45d7a7c3a4a6ae268ae6, State: Initialized, Role: FOLLOWER
I20260807 20:13:40.576232 8842 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebda05f9d98d45d7a7c3a4a6ae268ae6" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41671 } }
I20260807 20:13:40.576309 8842 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:13:40.576350 8842 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:13:40.576406 8842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:40.577353 8842 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebda05f9d98d45d7a7c3a4a6ae268ae6" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41671 } }
I20260807 20:13:40.577471 8842 leader_election.cc:302] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebda05f9d98d45d7a7c3a4a6ae268ae6; no voters:
I20260807 20:13:40.577625 8842 leader_election.cc:288] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:13:40.577670 8851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:40.577843 8851 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [term 1 LEADER]: Becoming Leader. State: Replica: ebda05f9d98d45d7a7c3a4a6ae268ae6, State: Running, Role: LEADER
I20260807 20:13:40.577870 8842 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:13:40.577987 8851 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebda05f9d98d45d7a7c3a4a6ae268ae6" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41671 } }
I20260807 20:13:40.578369 8853 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebda05f9d98d45d7a7c3a4a6ae268ae6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebda05f9d98d45d7a7c3a4a6ae268ae6" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41671 } } }
I20260807 20:13:40.578490 8853 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [sys.catalog]: This master's current role is: LEADER
I20260807 20:13:40.578637 8854 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebda05f9d98d45d7a7c3a4a6ae268ae6. Latest consensus state: current_term: 1 leader_uuid: "ebda05f9d98d45d7a7c3a4a6ae268ae6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebda05f9d98d45d7a7c3a4a6ae268ae6" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41671 } } }
I20260807 20:13:40.578694 8854 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6 [sys.catalog]: This master's current role is: LEADER
W20260807 20:13:40.579614 8877 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:13:40.579730 8877 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:13:40.579784 8867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:13:40.579948 8867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:13:40.581471 8867 catalog_manager.cc:1347] Generated new cluster ID: bc27c104158646099c8073602b3168d3
I20260807 20:13:40.581523 8867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:13:40 dist-test-slave-2j7r krb5kdc[8699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:40.585646 8850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:40.575946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37400 (local address 127.23.200.254:41671)
0807 20:13:40.576126 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:40.576136 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:40.576174 (+ 38us) server_negotiation.cc:407] Connection header received
0807 20:13:40.576288 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:40.576295 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:40.576341 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:40.576527 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:40.577237 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:40.578591 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:40.579690 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:40.580218 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:40.580324 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:40.582695 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:40.582699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:40.582699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:40.582712 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:40.584220 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:40.584544 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:40.584546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:40.584547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:40.584640 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:40.584821 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:40.584821 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:40.584822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:40.585063 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:40.585245 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:40.585314 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 20:13:40.585461 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
I20260807 20:13:40.617118 8867 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:13:40.617861 8867 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:13:40.636875 8867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6: Generated new TSK 0
I20260807 20:13:40.637154 8867 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:13:40.675736 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:41671
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:38519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:40.758159 8886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:40.758368 8886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:40.758389 8886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:40.759526 8886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:40.759568 8886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:40.759593 8886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:13:40.761124 8886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:38519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:41671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:40.761356 8886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:40.761581 8886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:40.766527 8892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:40.766527 8897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:40.766546 8886 server_base.cc:1034] running on GCE node
W20260807 20:13:40.766544 8895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:40.766872 8886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:40.767110 8886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:40.768256 8886 hybrid_clock.cc:648] HybridClock initialized: now 1786133620768243 us; error 29 us; skew 500 ppm
Aug 07 20:13:40 dist-test-slave-2j7r krb5kdc[8699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133620, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:40.770663 8886 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:13:40.771076 8886 webserver.cc:460] Webserver started at http://127.23.200.193:39275/ using document root <none> and password file <none>
I20260807 20:13:40.771279 8886 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:40.771327 8886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:40.771461 8886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:40.772531 8886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "9155b72203ef48dfa9f0f61158f52d63"
format_stamp: "Formatted at 2026-08-07 20:13:40 on dist-test-slave-2j7r"
server_key: "aeb6ca63bedf3190d99b15458a48c2a7"
server_key_iv: "315b8f929fbb0a0c1202b96d75725cea"
server_key_version: "encryptionkey@0"
I20260807 20:13:40.772856 8886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "9155b72203ef48dfa9f0f61158f52d63"
format_stamp: "Formatted at 2026-08-07 20:13:40 on dist-test-slave-2j7r"
server_key: "aeb6ca63bedf3190d99b15458a48c2a7"
server_key_iv: "315b8f929fbb0a0c1202b96d75725cea"
server_key_version: "encryptionkey@0"
I20260807 20:13:40.774169 8886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:13:40.774849 8907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:40.775014 8886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:40.775106 8886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "9155b72203ef48dfa9f0f61158f52d63"
format_stamp: "Formatted at 2026-08-07 20:13:40 on dist-test-slave-2j7r"
server_key: "aeb6ca63bedf3190d99b15458a48c2a7"
server_key_iv: "315b8f929fbb0a0c1202b96d75725cea"
server_key_version: "encryptionkey@0"
I20260807 20:13:40.775164 8886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:40.784899 8886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:40.786664 8886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:40.786794 8886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:40.787014 8886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:40.787297 8886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:40.787328 8886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:40.787356 8886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:40.787379 8886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:40.792934 8886 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:42393
I20260807 20:13:40.792994 9094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:42393 every 8 connection(s)
I20260807 20:13:40.793345 8886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
Aug 07 20:13:40 dist-test-slave-2j7r krb5kdc[8699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133620, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:40.801225 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 8886
I20260807 20:13:40.801317 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:13:40.801525 24355 external_mini_cluster.cc:1428] Setting key 849ce04994f51bbaf3b13f6fa062e88d
I20260807 20:13:40.801939 8850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:40.794070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:58329 (local address 127.23.200.254:41671)
0807 20:13:40.794128 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:40.794130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:40.794294 (+ 164us) server_negotiation.cc:407] Connection header received
0807 20:13:40.794758 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:40.794759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:40.794772 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:40.794827 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:40.795698 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:40.796209 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:40.797166 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:40.797362 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:40.797414 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:40.799338 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:40.799340 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:40.799340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:40.799347 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:40.800585 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:40.800895 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:40.800895 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:40.800896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:40.800958 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:40.801180 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:40.801181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:40.801182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:40.801300 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:40.801367 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:40.801727 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 20:13:40.801816 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 20:13:40.802073 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:40.794025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:41671 (local address 127.23.200.193:58329)
0807 20:13:40.794174 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:40.794216 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:40.794685 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:40.794882 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:40.794886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:40.795437 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:40.795630 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:40.795634 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:40.796275 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:40.796275 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:40.797080 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:40.797106 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:40.797452 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:40.797453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:40.797552 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:40.797952 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:40.797960 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:40.799259 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:40.800646 (+ 1387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:40.800649 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:40.800651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:40.800844 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:40.801002 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:40.801002 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:40.801003 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:40.801104 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:40.801391 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:40.801409 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:40.801575 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 20:13:40.801698 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 20:13:40.801857 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
I20260807 20:13:40.802536 9095 heartbeater.cc:347] Connected to a master server at 127.23.200.254:41671
I20260807 20:13:40.802628 9095 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:40.802801 9095 heartbeater.cc:511] Master 127.23.200.254:41671 requested a full tablet report, sending...
I20260807 20:13:40.803342 8775 ts_manager.cc:194] Registered new tserver with Master: 9155b72203ef48dfa9f0f61158f52d63 (127.23.200.193:42393)
I20260807 20:13:40.804170 8775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:58329
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:13:40.826246 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:41671
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:38519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:40.910909 9101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:40.911105 9101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:40.911127 9101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:40.912269 9101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:40.912312 9101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:40.912340 9101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:13:40.913810 9101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:38519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:41671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:40.914048 9101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:40.914266 9101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:40.919168 9109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:40.919227 9110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:40.919186 9101 server_base.cc:1034] running on GCE node
W20260807 20:13:40.919164 9112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:40.919611 9101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:40.919878 9101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:40.921033 9101 hybrid_clock.cc:648] HybridClock initialized: now 1786133620921020 us; error 28 us; skew 500 ppm
Aug 07 20:13:40 dist-test-slave-2j7r krb5kdc[8699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133620, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:40.923605 9101 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:13:40.924010 9101 webserver.cc:460] Webserver started at http://127.23.200.194:45967/ using document root <none> and password file <none>
I20260807 20:13:40.924230 9101 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:40.924278 9101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:40.924405 9101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:40.925494 9101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "b2f1e584a7334001a4f52d4265c45c05"
format_stamp: "Formatted at 2026-08-07 20:13:40 on dist-test-slave-2j7r"
server_key: "5ad37531e248e373352db78b4a4788d8"
server_key_iv: "07dd2405d8c32f6bf89358ccbbbabfc8"
server_key_version: "encryptionkey@0"
I20260807 20:13:40.925817 9101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "b2f1e584a7334001a4f52d4265c45c05"
format_stamp: "Formatted at 2026-08-07 20:13:40 on dist-test-slave-2j7r"
server_key: "5ad37531e248e373352db78b4a4788d8"
server_key_iv: "07dd2405d8c32f6bf89358ccbbbabfc8"
server_key_version: "encryptionkey@0"
I20260807 20:13:40.927016 9101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:13:40.927681 9129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:40.927825 9101 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:13:40.927917 9101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "b2f1e584a7334001a4f52d4265c45c05"
format_stamp: "Formatted at 2026-08-07 20:13:40 on dist-test-slave-2j7r"
server_key: "5ad37531e248e373352db78b4a4788d8"
server_key_iv: "07dd2405d8c32f6bf89358ccbbbabfc8"
server_key_version: "encryptionkey@0"
I20260807 20:13:40.927974 9101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:40.944851 9101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:40.946696 9101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:40.946839 9101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:40.947096 9101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:40.947448 9101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:40.947482 9101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:40.947512 9101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:40.947530 9101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:40.952939 9101 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:33365
I20260807 20:13:40.953006 9317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:33365 every 8 connection(s)
I20260807 20:13:40.953330 9101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
Aug 07 20:13:40 dist-test-slave-2j7r krb5kdc[8699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133620, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:40.961356 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 9101
I20260807 20:13:40.961449 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:13:40.961634 24355 external_mini_cluster.cc:1428] Setting key 70f95f1bc862c9591f079da1606da2f2
I20260807 20:13:40.962126 8850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:40.954027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:38723 (local address 127.23.200.254:41671)
0807 20:13:40.954088 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:40.954090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:40.954294 (+ 204us) server_negotiation.cc:407] Connection header received
0807 20:13:40.954761 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:40.954762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:40.954772 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:40.954827 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:40.955722 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:40.956213 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:40.957245 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:40.957439 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:40.957489 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:40.959531 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:40.959533 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:40.959533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:40.959540 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:40.960784 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:40.961167 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:40.961168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:40.961168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:40.961235 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:40.961478 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:40.961479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:40.961480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:40.961599 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:40.961656 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:40.961935 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 20:13:40.962027 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:13:40.962261 9324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:40.954004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:41671 (local address 127.23.200.194:38723)
0807 20:13:40.954165 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:40.954206 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:40.954681 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:40.954885 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:40.954889 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:40.955449 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:40.955640 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:40.955644 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:40.956348 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:40.956349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:40.957173 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:40.957175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:40.957498 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:40.957499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:40.957592 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:40.958012 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:40.958020 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:40.959444 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:40.960860 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:40.960866 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:40.960868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:40.961079 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:40.961298 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:40.961299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:40.961299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:40.961410 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:40.961656 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:40.961664 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:40.961812 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 20:13:40.961915 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 20:13:40.962068 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260807 20:13:40.962745 9322 heartbeater.cc:347] Connected to a master server at 127.23.200.254:41671
I20260807 20:13:40.962834 9322 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:40.963040 9322 heartbeater.cc:511] Master 127.23.200.254:41671 requested a full tablet report, sending...
I20260807 20:13:40.963464 8775 ts_manager.cc:194] Registered new tserver with Master: b2f1e584a7334001a4f52d4265c45c05 (127.23.200.194:33365)
I20260807 20:13:40.963897 8775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:38723
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:13:40.987951 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:41671
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:38519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:13:41.069792 9330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:13:41.070010 9330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:13:41.070035 9330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:13:41.071132 9330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:13:41.071167 9330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:13:41.071192 9330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:13:41.072602 9330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:38519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:41671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:13:41.072816 9330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:13:41.073050 9330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:13:41.077919 9338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:41.077932 9337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:13:41.077957 9342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:13:41.078261 9330 server_base.cc:1034] running on GCE node
I20260807 20:13:41.078496 9330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:13:41.078727 9330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:13:41.079861 9330 hybrid_clock.cc:648] HybridClock initialized: now 1786133621079845 us; error 33 us; skew 500 ppm
Aug 07 20:13:41 dist-test-slave-2j7r krb5kdc[8699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133621, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:13:41.082365 9330 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:13:41.082787 9330 webserver.cc:460] Webserver started at http://127.23.200.195:40995/ using document root <none> and password file <none>
I20260807 20:13:41.082964 9330 fs_manager.cc:359] Metadata directory not provided
I20260807 20:13:41.083005 9330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:13:41.083101 9330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:13:41.084111 9330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "016cc13ff50d42258c7a91553f420838"
format_stamp: "Formatted at 2026-08-07 20:13:41 on dist-test-slave-2j7r"
server_key: "44960be0df8f0e81c584d9fbf91d9cbb"
server_key_iv: "8b3fcbdd05cd2485a41c5303991b2bf2"
server_key_version: "encryptionkey@0"
I20260807 20:13:41.084415 9330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "016cc13ff50d42258c7a91553f420838"
format_stamp: "Formatted at 2026-08-07 20:13:41 on dist-test-slave-2j7r"
server_key: "44960be0df8f0e81c584d9fbf91d9cbb"
server_key_iv: "8b3fcbdd05cd2485a41c5303991b2bf2"
server_key_version: "encryptionkey@0"
I20260807 20:13:41.085618 9330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 20:13:41.086290 9351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:41.086452 9330 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:41.086535 9330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "016cc13ff50d42258c7a91553f420838"
format_stamp: "Formatted at 2026-08-07 20:13:41 on dist-test-slave-2j7r"
server_key: "44960be0df8f0e81c584d9fbf91d9cbb"
server_key_iv: "8b3fcbdd05cd2485a41c5303991b2bf2"
server_key_version: "encryptionkey@0"
I20260807 20:13:41.086589 9330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:13:41.099606 9330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:13:41.101503 9330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:13:41.101611 9330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:13:41.101791 9330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:13:41.102061 9330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:13:41.102090 9330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:41.102111 9330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:13:41.102123 9330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:13:41.107507 9330 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:46165
I20260807 20:13:41.107582 9535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:46165 every 8 connection(s)
I20260807 20:13:41.107841 9330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:13:41.113535 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 9330
I20260807 20:13:41.113618 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
Aug 07 20:13:41 dist-test-slave-2j7r krb5kdc[8699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133621, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:13:41.113783 24355 external_mini_cluster.cc:1428] Setting key 6ebc21caf5a524abefaef3d1d337b691
I20260807 20:13:41.116469 8850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:41.108531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:38691 (local address 127.23.200.254:41671)
0807 20:13:41.108594 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:41.108596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:41.108809 (+ 213us) server_negotiation.cc:407] Connection header received
0807 20:13:41.109351 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:41.109352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:41.109363 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:41.109426 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:41.110346 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.110844 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.111834 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.112034 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.112096 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:41.114113 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:41.114114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:41.114115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:41.114122 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:41.115392 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:41.115746 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:41.115747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:41.115747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:41.115808 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:41.115991 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:41.115991 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:41.115991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:41.116088 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:41.116132 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:41.116324 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 20:13:41.116389 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:13:41.116612 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:41.108493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:41671 (local address 127.23.200.195:38691)
0807 20:13:41.108655 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:41.108695 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:41.109217 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:41.109515 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:41.109519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:41.110069 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:13:41.110252 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:41.110256 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.110927 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.110928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:41.111742 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:41.111744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.112101 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.112102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:41.112195 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:41.112662 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:13:41.112671 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:13:41.114008 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:13:41.115471 (+ 1463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:41.115478 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:41.115479 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:41.115681 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:41.115862 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:13:41.115862 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:13:41.115862 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:13:41.115948 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:13:41.116123 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:13:41.116125 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:13:41.116232 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 20:13:41.116306 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 20:13:41.116428 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260807 20:13:41.117064 9536 heartbeater.cc:347] Connected to a master server at 127.23.200.254:41671
I20260807 20:13:41.117182 9536 heartbeater.cc:464] Registering TS with master...
I20260807 20:13:41.117357 9536 heartbeater.cc:511] Master 127.23.200.254:41671 requested a full tablet report, sending...
I20260807 20:13:41.117767 8775 ts_manager.cc:194] Registered new tserver with Master: 016cc13ff50d42258c7a91553f420838 (127.23.200.195:46165)
I20260807 20:13:41.118126 8775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:38691
I20260807 20:13:41.125532 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:13:41.132421 8850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:41.126753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37414 (local address 127.23.200.254:41671)
0807 20:13:41.126796 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:41.126797 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:41.126884 (+ 87us) server_negotiation.cc:407] Connection header received
0807 20:13:41.126955 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:41.126956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:41.126964 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:41.127011 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:13:41.127614 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.128080 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.128830 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.129002 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.129041 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:41.130169 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:13:41.130170 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:13:41.130170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:13:41.130175 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:13:41.131373 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:41.131733 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:41.131734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:41.131734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:41.131791 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:13:41.131989 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:13:41.131989 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:13:41.131989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:13:41.132130 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:13:41.132200 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:41.132277 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 20:13:41.132337 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:13:41.133884 8775 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:13:41.135484 8775 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:13:41.141602 9560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:41.137748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43030 (local address 127.23.200.195:46165)
0807 20:13:41.137955 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:41.137958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:41.138276 (+ 318us) server_negotiation.cc:407] Connection header received
0807 20:13:41.138325 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:41.138327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:41.138342 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:41.138411 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:41.138589 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.139615 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.140652 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.141221 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.141287 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:41.141346 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:41.141481 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 20:13:41.141517 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":97,"threads_started":1}
I20260807 20:13:41.141592 9555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:41.137512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:46165 (local address 127.0.0.1:43030)
0807 20:13:41.138131 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:41.138139 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:41.138179 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:41.138479 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:41.138480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:41.138482 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:41.138514 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:41.138515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.139702 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.139703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:41.140571 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:41.140574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.141293 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.141294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:41.141373 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:41.141387 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 20:13:41.141460 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 20:13:41.141516 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":78,"threads_started":1}
I20260807 20:13:41.141762 9556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:41.137750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56142 (local address 127.23.200.193:42393)
0807 20:13:41.137922 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:41.137928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:41.138063 (+ 135us) server_negotiation.cc:407] Connection header received
0807 20:13:41.138125 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:41.138128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:41.138144 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:41.138237 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:41.138468 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.139187 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.140538 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.141275 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.141346 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:41.141409 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:41.141647 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 20:13:41.141689 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260807 20:13:41.141713 9552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:41.137502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:33365 (local address 127.0.0.1:54502)
0807 20:13:41.137614 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:41.137657 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:41.137726 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:41.138114 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:41.138117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:41.138120 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:41.138187 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:41.138189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.139621 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.139623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:41.140654 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:41.140656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.141534 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.141534 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:41.141587 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:41.141590 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:13:41.141616 (+ 26us) client_negotiation.cc:240] Negotiation successful
0807 20:13:41.141648 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":81,"threads_started":1}
I20260807 20:13:41.142027 9558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:41.137750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54502 (local address 127.23.200.194:33365)
0807 20:13:41.137957 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:41.137961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:41.137971 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:13:41.138002 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:41.138004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:41.138016 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:41.138092 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:41.138318 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.139506 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.140720 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.141447 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.141789 (+ 342us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:41.141910 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:41.141933 (+ 23us) server_negotiation.cc:299] Negotiation successful
0807 20:13:41.141967 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":85,"threads_started":1}
I20260807 20:13:41.141995 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:41.137502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:42393 (local address 127.0.0.1:56142)
0807 20:13:41.137995 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:41.138004 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:41.138057 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:41.138325 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:41.138326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:41.138328 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:41.138383 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:41.138385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.139273 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.139274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:41.140456 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:41.140459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.141449 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.141450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:41.141515 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:41.141519 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:13:41.141563 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 20:13:41.141603 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":6,"thread_start_us":60,"threads_started":1}
I20260807 20:13:41.142798 8996 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb88f2a6b76487cb00ecc96f144de57 (DEFAULT_TABLE table=test-table [id=996d1b7aa00043e289d9c1bc3a917186]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:41.142855 9206 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb88f2a6b76487cb00ecc96f144de57 (DEFAULT_TABLE table=test-table [id=996d1b7aa00043e289d9c1bc3a917186]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:41.142987 9436 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb88f2a6b76487cb00ecc96f144de57 (DEFAULT_TABLE table=test-table [id=996d1b7aa00043e289d9c1bc3a917186]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:13:41.143098 8996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb88f2a6b76487cb00ecc96f144de57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:41.143195 9206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb88f2a6b76487cb00ecc96f144de57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:41.143271 9436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb88f2a6b76487cb00ecc96f144de57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:13:41.145843 9561 tablet_bootstrap.cc:492] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63: Bootstrap starting.
I20260807 20:13:41.146376 9563 tablet_bootstrap.cc:492] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05: Bootstrap starting.
I20260807 20:13:41.146597 9561 tablet_bootstrap.cc:654] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:41.146612 9564 tablet_bootstrap.cc:492] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838: Bootstrap starting.
I20260807 20:13:41.147197 9561 log.cc:824] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:41.147233 9564 tablet_bootstrap.cc:654] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:41.147830 9564 log.cc:824] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:41.148409 9561 tablet_bootstrap.cc:492] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63: No bootstrap required, opened a new log
I20260807 20:13:41.148466 9561 ts_tablet_manager.cc:1397] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:13:41.148552 9564 tablet_bootstrap.cc:492] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838: No bootstrap required, opened a new log
I20260807 20:13:41.148617 9564 ts_tablet_manager.cc:1397] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:13:41.149866 9561 raft_consensus.cc:348] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f1e584a7334001a4f52d4265c45c05" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33365 } } peers { permanent_uuid: "9155b72203ef48dfa9f0f61158f52d63" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42393 } } peers { permanent_uuid: "016cc13ff50d42258c7a91553f420838" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46165 } }
I20260807 20:13:41.150012 9561 raft_consensus.cc:374] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:41.150035 9561 raft_consensus.cc:729] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9155b72203ef48dfa9f0f61158f52d63, State: Initialized, Role: FOLLOWER
I20260807 20:13:41.149991 9564 raft_consensus.cc:348] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f1e584a7334001a4f52d4265c45c05" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33365 } } peers { permanent_uuid: "9155b72203ef48dfa9f0f61158f52d63" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42393 } } peers { permanent_uuid: "016cc13ff50d42258c7a91553f420838" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46165 } }
I20260807 20:13:41.150148 9564 raft_consensus.cc:374] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:41.150136 9561 consensus_queue.cc:260] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f1e584a7334001a4f52d4265c45c05" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33365 } } peers { permanent_uuid: "9155b72203ef48dfa9f0f61158f52d63" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42393 } } peers { permanent_uuid: "016cc13ff50d42258c7a91553f420838" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46165 } }
I20260807 20:13:41.150192 9564 raft_consensus.cc:729] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016cc13ff50d42258c7a91553f420838, State: Initialized, Role: FOLLOWER
I20260807 20:13:41.150303 9564 consensus_queue.cc:260] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f1e584a7334001a4f52d4265c45c05" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33365 } } peers { permanent_uuid: "9155b72203ef48dfa9f0f61158f52d63" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42393 } } peers { permanent_uuid: "016cc13ff50d42258c7a91553f420838" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46165 } }
I20260807 20:13:41.150408 9561 ts_tablet_manager.cc:1428] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:13:41.150439 9095 heartbeater.cc:503] Master 127.23.200.254:41671 was elected leader, sending a full tablet report...
I20260807 20:13:41.150586 9563 tablet_bootstrap.cc:654] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05: Neither blocks nor log segments found. Creating new log.
I20260807 20:13:41.150589 9536 heartbeater.cc:503] Master 127.23.200.254:41671 was elected leader, sending a full tablet report...
I20260807 20:13:41.150832 9564 ts_tablet_manager.cc:1428] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:13:41.151170 9563 log.cc:824] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05: Log is configured to *not* fsync() on all Append() calls
I20260807 20:13:41.151904 9563 tablet_bootstrap.cc:492] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05: No bootstrap required, opened a new log
I20260807 20:13:41.151988 9563 ts_tablet_manager.cc:1397] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 20:13:41.153386 9563 raft_consensus.cc:348] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f1e584a7334001a4f52d4265c45c05" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33365 } } peers { permanent_uuid: "9155b72203ef48dfa9f0f61158f52d63" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42393 } } peers { permanent_uuid: "016cc13ff50d42258c7a91553f420838" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46165 } }
I20260807 20:13:41.153509 9563 raft_consensus.cc:374] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:13:41.153540 9563 raft_consensus.cc:729] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2f1e584a7334001a4f52d4265c45c05, State: Initialized, Role: FOLLOWER
I20260807 20:13:41.153645 9563 consensus_queue.cc:260] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f1e584a7334001a4f52d4265c45c05" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33365 } } peers { permanent_uuid: "9155b72203ef48dfa9f0f61158f52d63" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42393 } } peers { permanent_uuid: "016cc13ff50d42258c7a91553f420838" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46165 } }
I20260807 20:13:41.153867 9563 ts_tablet_manager.cc:1428] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:13:41.153910 9322 heartbeater.cc:503] Master 127.23.200.254:41671 was elected leader, sending a full tablet report...
I20260807 20:13:41.185627 9568 raft_consensus.cc:482] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:13:41.185772 9568 raft_consensus.cc:504] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f1e584a7334001a4f52d4265c45c05" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33365 } } peers { permanent_uuid: "9155b72203ef48dfa9f0f61158f52d63" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42393 } } peers { permanent_uuid: "016cc13ff50d42258c7a91553f420838" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46165 } }
I20260807 20:13:41.186182 9568 leader_election.cc:288] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2f1e584a7334001a4f52d4265c45c05 (127.23.200.194:33365), 016cc13ff50d42258c7a91553f420838 (127.23.200.195:46165)
I20260807 20:13:41.189110 9558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:41.186593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:38429 (local address 127.23.200.194:33365)
0807 20:13:41.186639 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:41.186640 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:41.186672 (+ 32us) server_negotiation.cc:407] Connection header received
0807 20:13:41.186747 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:41.186748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:41.186759 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:41.186812 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:41.186963 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.187395 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.188368 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.188791 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.188863 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:41.188871 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:41.188995 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 20:13:41.189030 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:13:41.189116 9560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:41.186558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:42739 (local address 127.23.200.195:46165)
0807 20:13:41.186604 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:41.186606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:41.186615 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:41.186644 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:41.186645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:41.186654 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:41.186704 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:13:41.186856 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.187391 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.188336 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.188760 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.188820 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:41.188828 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:41.188979 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 20:13:41.189014 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:13:41.189110 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:41.186371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:46165 (local address 127.23.200.193:42739)
0807 20:13:41.186447 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:41.186459 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:41.186507 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:41.186711 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:41.186713 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:41.186714 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:41.186791 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:41.186793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.187476 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.187477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:41.188248 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:41.188250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.188826 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.188828 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:41.188914 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:41.188918 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:13:41.188966 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 20:13:41.189007 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 20:13:41.189129 9574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:41.186424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:33365 (local address 127.23.200.193:38429)
0807 20:13:41.186614 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:41.186624 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:13:41.186684 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:13:41.186824 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:13:41.186825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:13:41.186827 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:13:41.186902 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:41.186903 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.187487 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.187488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:41.188248 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:13:41.188250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.188870 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.188871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:13:41.188928 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:41.188931 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:13:41.188979 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 20:13:41.189015 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":33,"thread_start_us":125,"threads_started":1}
I20260807 20:13:41.189450 9467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb88f2a6b76487cb00ecc96f144de57" candidate_uuid: "9155b72203ef48dfa9f0f61158f52d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016cc13ff50d42258c7a91553f420838" is_pre_election: true
I20260807 20:13:41.189625 9467 raft_consensus.cc:2454] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9155b72203ef48dfa9f0f61158f52d63 in term 0.
I20260807 20:13:41.189693 9237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb88f2a6b76487cb00ecc96f144de57" candidate_uuid: "9155b72203ef48dfa9f0f61158f52d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2f1e584a7334001a4f52d4265c45c05" is_pre_election: true
I20260807 20:13:41.189847 9237 raft_consensus.cc:2454] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9155b72203ef48dfa9f0f61158f52d63 in term 0.
I20260807 20:13:41.189848 8909 leader_election.cc:302] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016cc13ff50d42258c7a91553f420838, 9155b72203ef48dfa9f0f61158f52d63; no voters:
I20260807 20:13:41.189986 9568 raft_consensus.cc:2781] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:13:41.190060 9568 raft_consensus.cc:482] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:13:41.190083 9568 raft_consensus.cc:3037] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:41.190788 9568 raft_consensus.cc:504] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f1e584a7334001a4f52d4265c45c05" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33365 } } peers { permanent_uuid: "9155b72203ef48dfa9f0f61158f52d63" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42393 } } peers { permanent_uuid: "016cc13ff50d42258c7a91553f420838" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46165 } }
I20260807 20:13:41.190908 9568 leader_election.cc:288] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [CANDIDATE]: Term 1 election: Requested vote from peers b2f1e584a7334001a4f52d4265c45c05 (127.23.200.194:33365), 016cc13ff50d42258c7a91553f420838 (127.23.200.195:46165)
I20260807 20:13:41.191066 9467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb88f2a6b76487cb00ecc96f144de57" candidate_uuid: "9155b72203ef48dfa9f0f61158f52d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016cc13ff50d42258c7a91553f420838"
I20260807 20:13:41.191074 9237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb88f2a6b76487cb00ecc96f144de57" candidate_uuid: "9155b72203ef48dfa9f0f61158f52d63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2f1e584a7334001a4f52d4265c45c05"
I20260807 20:13:41.191131 9467 raft_consensus.cc:3037] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:41.191143 9237 raft_consensus.cc:3037] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:13:41.191833 9467 raft_consensus.cc:2454] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9155b72203ef48dfa9f0f61158f52d63 in term 1.
I20260807 20:13:41.191948 9237 raft_consensus.cc:2454] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9155b72203ef48dfa9f0f61158f52d63 in term 1.
I20260807 20:13:41.192013 8909 leader_election.cc:302] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016cc13ff50d42258c7a91553f420838, 9155b72203ef48dfa9f0f61158f52d63; no voters:
I20260807 20:13:41.192096 9568 raft_consensus.cc:2781] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:13:41.192220 9568 raft_consensus.cc:686] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [term 1 LEADER]: Becoming Leader. State: Replica: 9155b72203ef48dfa9f0f61158f52d63, State: Running, Role: LEADER
I20260807 20:13:41.192301 9568 consensus_queue.cc:237] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f1e584a7334001a4f52d4265c45c05" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33365 } } peers { permanent_uuid: "9155b72203ef48dfa9f0f61158f52d63" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42393 } } peers { permanent_uuid: "016cc13ff50d42258c7a91553f420838" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46165 } }
I20260807 20:13:41.193202 8774 catalog_manager.cc:5495] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9155b72203ef48dfa9f0f61158f52d63 (127.23.200.193). New cstate: current_term: 1 leader_uuid: "9155b72203ef48dfa9f0f61158f52d63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2f1e584a7334001a4f52d4265c45c05" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9155b72203ef48dfa9f0f61158f52d63" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 42393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "016cc13ff50d42258c7a91553f420838" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260807 20:13:41.202603 9556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:41.200402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56156 (local address 127.23.200.193:42393)
0807 20:13:41.200487 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 20:13:41.200489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:13:41.200498 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:13:41.200534 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:13:41.200535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:13:41.200553 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:13:41.200633 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:13:41.200753 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.201265 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.201990 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:13:41.202180 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:13:41.202237 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:13:41.202333 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:13:41.202439 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:13:41.202496 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 20:13:41.202510 (+ 14us) server_negotiation.cc:299] Negotiation successful
0807 20:13:41.202527 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 20:13:41.203828 9323 tablet.cc:2369] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:13:41.204341 9237 raft_consensus.cc:1261] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05 [term 1 FOLLOWER]: Refusing update from remote peer 9155b72203ef48dfa9f0f61158f52d63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:41.204348 9467 raft_consensus.cc:1261] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838 [term 1 FOLLOWER]: Refusing update from remote peer 9155b72203ef48dfa9f0f61158f52d63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:13:41.204715 9576 consensus_queue.cc:1035] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2f1e584a7334001a4f52d4265c45c05" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:41.204854 9568 consensus_queue.cc:1035] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "016cc13ff50d42258c7a91553f420838" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:13:41.207734 9585 mvcc.cc:205] Tried to move back new op lower bound from 7316003312450400256 to 7316003312404262912. Current Snapshot: MvccSnapshot[applied={T|T < 7316003312450400256 or (T in {7316003312450400256})}]
I20260807 20:13:41.209509 8774 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:41.209599 8774 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:13:41.210265 8774 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ebda05f9d98d45d7a7c3a4a6ae268ae6: Sending DeleteTablet for 3 replicas of tablet 7eb88f2a6b76487cb00ecc96f144de57
I20260807 20:13:41.210582 8996 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb88f2a6b76487cb00ecc96f144de57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:41 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:56142
I20260807 20:13:41.210570 9206 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb88f2a6b76487cb00ecc96f144de57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:41 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:54502
I20260807 20:13:41.210700 9436 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb88f2a6b76487cb00ecc96f144de57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:13:41 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:43030
I20260807 20:13:41.210807 9596 tablet_replica.cc:330] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63: stopping tablet replica
I20260807 20:13:41.210907 9603 tablet_replica.cc:330] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838: stopping tablet replica
I20260807 20:13:41.210903 9596 raft_consensus.cc:2229] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:13:41.210987 9603 raft_consensus.cc:2229] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:41.211005 9596 raft_consensus.cc:2258] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:41.211244 9603 raft_consensus.cc:2258] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:41.211269 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 8886
I20260807 20:13:41.211385 9596 ts_tablet_manager.cc:1905] T 7eb88f2a6b76487cb00ecc96f144de57 P 9155b72203ef48dfa9f0f61158f52d63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:41.214514 9603 ts_tablet_manager.cc:1905] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:41.216373 9603 ts_tablet_manager.cc:1918] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:13:41.216465 9603 log.cc:1196] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/7eb88f2a6b76487cb00ecc96f144de57
I20260807 20:13:41.216727 9603 ts_tablet_manager.cc:1939] T 7eb88f2a6b76487cb00ecc96f144de57 P 016cc13ff50d42258c7a91553f420838: Deleting consensus metadata
I20260807 20:13:41.217262 8750 catalog_manager.cc:4841] TS 016cc13ff50d42258c7a91553f420838 (127.23.200.195:46165): tablet 7eb88f2a6b76487cb00ecc96f144de57 (table test-table [id=996d1b7aa00043e289d9c1bc3a917186]) successfully deleted
I20260807 20:13:41.218943 9602 tablet_replica.cc:330] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05: stopping tablet replica
I20260807 20:13:41.219071 9602 raft_consensus.cc:2229] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:13:41.219195 9602 raft_consensus.cc:2258] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:13:41.219583 9602 ts_tablet_manager.cc:1905] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:13:41.221004 9602 ts_tablet_manager.cc:1918] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:13:41.221073 9602 log.cc:1196] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/7eb88f2a6b76487cb00ecc96f144de57
I20260807 20:13:41.221340 9602 ts_tablet_manager.cc:1939] T 7eb88f2a6b76487cb00ecc96f144de57 P b2f1e584a7334001a4f52d4265c45c05: Deleting consensus metadata
I20260807 20:13:41.221768 8751 catalog_manager.cc:4841] TS b2f1e584a7334001a4f52d4265c45c05 (127.23.200.194:33365): tablet 7eb88f2a6b76487cb00ecc96f144de57 (table test-table [id=996d1b7aa00043e289d9c1bc3a917186]) successfully deleted
W20260807 20:13:41.223235 8749 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:42393 (error 108)
I20260807 20:13:41.223719 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:13:41.223595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:42393 (local address 127.0.0.1:56170)
0807 20:13:41.223648 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 20:13:41.223686 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:42393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 20:13:41.223805 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 9101
W20260807 20:13:41.223867 8749 catalog_manager.cc:4584] TS 9155b72203ef48dfa9f0f61158f52d63 (127.23.200.193:42393): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7eb88f2a6b76487cb00ecc96f144de57: Network error: Client connection negotiation failed: client connection to 127.23.200.193:42393: connect: Connection refused (error 111)
I20260807 20:13:41.244331 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 9330
I20260807 20:13:41.254153 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 8724
2026-08-07T20:13:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2092 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:13:41 dist-test-slave-2j7r krb5kdc[9610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:13:41 dist-test-slave-2j7r krb5kdc[9610](info): set up 2 sockets
Aug 07 20:13:41 dist-test-slave-2j7r krb5kdc[9610](info): commencing operation
krb5kdc: starting...
W20260807 20:13:42.424634 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:13:42 dist-test-slave-2j7r krb5kdc[9610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:13:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:13:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.23.200.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.23.200.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.23.200.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/rangeradmin_127.23.200.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.23.200.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/rangeradmin_127.23.200.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.23.200.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.23.200.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.23.200.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/rangerlookup_127.23.200.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.23.200.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/rangerlookup_127.23.200.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.200.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.23.200.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.200.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/HTTP_127.23.200.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.200.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/HTTP_127.23.200.212@KRBTEST.COM.keytab.
I20260807 20:13:42.598368 24355 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskISIRft/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/postgres -l logfile start
2026-08-07 20:13:45.143 UTC [9655] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 20:13:45.144 UTC [9655] LOG: listening on IPv4 address "127.23.200.212", port 36869
2026-08-07 20:13:45.147 UTC [9655] LOG: listening on Unix socket "/tmp/.s.PGSQL.36869"
2026-08-07 20:13:45.176 UTC [9662] LOG: database system was shut down at 2026-08-07 20:13:43 UTC
2026-08-07 20:13:45.180 UTC [9655] LOG: database system is ready to accept connections
I20260807 20:13:46.249742 24355 mini_postgres.cc:96] Postgres bound to 36869
2026-08-07 20:13:46.257 UTC [9678] FATAL: database "slave" does not exist
127.23.200.212:36869 - accepting connections
I20260807 20:13:46.258647 24355 mini_ranger.cc:161] Starting Ranger...
I20260807 20:13:46.274255 24355 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 20:13:46.767740 24355 mini_ranger.cc:87] Created ranger Postgres database
I20260807 20:13:46.767817 24355 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-admin
2026-08-07 20:13:47,058 [I] DB FLAVOR :POSTGRES
2026-08-07 20:13:47,058 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 20:13:47,059 [I] Checking connection..
2026-08-07 20:13:47,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 20:13:47,330 [I] Checking connection passed.
2026-08-07 20:13:47,330 [I] --------- Verifying version history table ---------
2026-08-07 20:13:47,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 20:13:47,598 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 20:13:47,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 20:13:47,867 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 20:13:47,867 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 20:13:47,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 20:13:48.103 UTC [9774] WARNING: there is no transaction in progress
2026-08-07 20:13:48,111 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 20:13:48,112 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 20:13:48,372 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 20:13:48,373 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 20:13:48,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 20:13:48,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 20:13:48,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2j7r.c.gcp-upstream.internal','N') ;"
2026-08-07 20:13:49,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 20:13:49,424 [I] Table x_portal_user does not exist in database ranger
2026-08-07 20:13:49,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 20:13:49,677 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 20:13:49,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 20:13:49,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 20:13:50,192 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 20:13:50,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 20:13:51.165 UTC [9931] WARNING: there is no transaction in progress
2026-08-07 20:13:51.180 UTC [9931] WARNING: there is no transaction in progress
2026-08-07 20:13:51.195 UTC [9931] WARNING: there is no transaction in progress
2026-08-07 20:13:51.211 UTC [9931] WARNING: there is no transaction in progress
2026-08-07 20:13:51.227 UTC [9931] WARNING: there is no transaction in progress
2026-08-07 20:13:51.248 UTC [9931] WARNING: there is no transaction in progress
2026-08-07 20:13:51.309 UTC [9931] WARNING: there is no transaction in progress
2026-08-07 20:13:51.316 UTC [9931] WARNING: there is no transaction in progress
2026-08-07 20:13:51.330 UTC [9931] WARNING: there is no transaction in progress
2026-08-07 20:13:51.340 UTC [9931] WARNING: there is no transaction in progress
2026-08-07 20:13:51.351 UTC [9931] WARNING: there is no transaction in progress
2026-08-07 20:13:51.366 UTC [9931] WARNING: there is no transaction in progress
2026-08-07 20:13:51.376 UTC [9931] WARNING: there is no transaction in progress
2026-08-07 20:13:51.382 UTC [9931] WARNING: there is no transaction in progress
2026-08-07 20:13:51.389 UTC [9931] WARNING: there is no transaction in progress
2026-08-07 20:13:51.396 UTC [9931] WARNING: there is no transaction in progress
2026-08-07 20:13:52,040 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 20:13:52,041 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 20:13:52,300 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 20:13:52,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 20:13:52,567 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 20:13:52,567 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 20:13:52,845 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 20:13:52,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 20:13:53,111 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 20:13:53,344 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2j7r.c.gcp-upstream.internal';"
2026-08-07 20:13:53,603 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 20:13:53,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 20:13:53,865 [I] DB_PATCHES have already been applied
I20260807 20:13:53.870987 24355 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-admin:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskISIRft/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 20:13:53.871054 24355 mini_ranger.cc:193] Using host: 127.23.200.212
I20260807 20:13:53.875147 24355 mini_ranger.cc:227] Ranger admin URL: http://127.23.200.212:40625
Aug 07, 2026 8:13:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:13:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:13:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 8:13:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 8:13:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 8:13:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 8:13:54 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 20:13:55.110775 24355 mini_ranger.cc:160] Time spent starting Ranger: real 8.852s user 0.006s sys 0.005s
Aug 07, 2026 8:14:15 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 8:14:15 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 8:14:15 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 8:14:15 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 8:14:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 8:14:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 8:14:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 8:14:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 8:14:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 8:14:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 8:14:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 8:14:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 8:14:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 8:14:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 8:14:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 8:14:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 8:14:15 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 8:14:16 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 20:14:16.830861 24355 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.23.200.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.23.200.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.23.200.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/rangerkms_127.23.200.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.23.200.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/rangerkms_127.23.200.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.200.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.23.200.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.200.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/HTTP_127.23.200.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.200.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/HTTP_127.23.200.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 20:14:16 dist-test-slave-2j7r krb5kdc[9610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133656, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 20:14:16.949517 24355 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 20:14:16.960963 24355 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 20:14:17.828996 24355 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 20:14:17.829066 24355 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-kms
I20260807 20:14:17.829078 24355 mini_ranger_kms.cc:227] Using postgres at 127.23.200.212:36869
2026-08-07 20:14:18,417 [I] DB FLAVOR :POSTGRES
2026-08-07 20:14:18,417 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 20:14:18,417 [I] Checking connection
2026-08-07 20:14:18,417 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 20:14:18,654 [I] connection success
2026-08-07 20:14:18,654 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 20:14:18,654 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 20:14:18,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 20:14:18,927 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 20:14:18,927 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 20:14:18,928 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 20:14:18,928 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:36869/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 20:14:19,183 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 20:14:19.625413 24355 mini_ranger_kms.cc:310] Created kms service
I20260807 20:14:19.790921 24355 mini_ranger_kms.cc:326] Created kudu user
I20260807 20:14:19.833231 24355 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 20:14:19 dist-test-slave-2j7r krb5kdc[9610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133656, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
I20260807 20:14:20.199451 24355 mini_ranger_kms.cc:384] Added ranger policy
I20260807 20:14:20.199568 24355 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-kms:/tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 20:14:20.199590 24355 mini_ranger_kms.cc:240] Using host: 127.23.200.212
I20260807 20:14:20.203785 24355 mini_ranger_kms.cc:276] Ranger KMS PID: 10215
I20260807 20:14:20.203930 24355 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.23.200.212:52929
Aug 07, 2026 8:14:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:14:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:14:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 8:14:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 8:14:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 8:14:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 20:14:22 dist-test-slave-2j7r krb5kdc[9610](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786133662, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.200.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:14:22.746248 24355 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.797s user 0.010s sys 0.213s
I20260807 20:14:22.746347 24355 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 20:14:22.746428 24355 mini_ranger_kms.cc:401] 127.23.200.212:52929/kms/v1/keys
Aug 07 20:14:23 dist-test-slave-2j7r krb5kdc[9610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133662, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.200.212@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
Aug 07 20:14:23 dist-test-slave-2j7r krb5kdc[9610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133662, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.200.212@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
Aug 07 20:14:25 dist-test-slave-2j7r krb5kdc[9610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:14:25.719985 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:39715
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:39715
--ranger_config_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.200.212:52929/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:14:25.815075 10302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:14:25.815295 10302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:14:25.815317 10302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:14:25.816644 10302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:14:25.816699 10302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:14:25.816711 10302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:14:25.816723 10302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:14:25.816735 10302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:14:25.818447 10302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/wal
--ranger_kms_url=127.23.200.212:52929/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:39715
--ranger_config_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:39715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:14:25.818645 10302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:14:25.818820 10302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:14:25.823938 10313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:14:25.823956 10315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:14:25.824096 10309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:14:25.824244 10302 server_base.cc:1034] running on GCE node
I20260807 20:14:25.824456 10302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:14:25.824738 10302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:14:25.825872 10302 hybrid_clock.cc:648] HybridClock initialized: now 1786133665825857 us; error 33 us; skew 500 ppm
Aug 07 20:14:25 dist-test-slave-2j7r krb5kdc[9610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133665, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:14:25.832780 10302 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:14:25.833307 10302 webserver.cc:460] Webserver started at http://127.23.200.254:42595/ using document root <none> and password file <none>
I20260807 20:14:25.833557 10302 fs_manager.cc:359] Metadata directory not provided
I20260807 20:14:25.833607 10302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:14:25.833775 10302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 20:14:25 dist-test-slave-2j7r krb5kdc[9610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133665, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
I20260807 20:14:25.895984 10302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "355ae27b925f4710b0b37763bd90d164"
format_stamp: "Formatted at 2026-08-07 20:14:25 on dist-test-slave-2j7r"
server_key: "3bfe9b5a5b0ec60082ccf26b9cd9bead"
server_key_iv: "bdafb6f89381cf0252d361cba1dec5f9"
server_key_version: "kuduclusterkey@0"
I20260807 20:14:25.896358 10302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "355ae27b925f4710b0b37763bd90d164"
format_stamp: "Formatted at 2026-08-07 20:14:25 on dist-test-slave-2j7r"
server_key: "3bfe9b5a5b0ec60082ccf26b9cd9bead"
server_key_iv: "bdafb6f89381cf0252d361cba1dec5f9"
server_key_version: "kuduclusterkey@0"
I20260807 20:14:25.897894 10302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260807 20:14:25.909572 10325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:14:25.909775 10302 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:14:25.909886 10302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "355ae27b925f4710b0b37763bd90d164"
format_stamp: "Formatted at 2026-08-07 20:14:25 on dist-test-slave-2j7r"
server_key: "3bfe9b5a5b0ec60082ccf26b9cd9bead"
server_key_iv: "bdafb6f89381cf0252d361cba1dec5f9"
server_key_version: "kuduclusterkey@0"
I20260807 20:14:25.909984 10302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:14:25.944671 10302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:14:25.946714 10302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:14:25.946885 10302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:14:25.951656 10302 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:39715
I20260807 20:14:25.951720 10395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:39715 every 8 connection(s)
I20260807 20:14:25.952101 10302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:14:25.952801 10397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:14:25.955149 10397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164: Bootstrap starting.
I20260807 20:14:25.955734 10397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164: Neither blocks nor log segments found. Creating new log.
I20260807 20:14:25.956019 10397 log.cc:824] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164: Log is configured to *not* fsync() on all Append() calls
I20260807 20:14:25.956671 10397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164: No bootstrap required, opened a new log
I20260807 20:14:25.957886 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 10302
I20260807 20:14:25.958019 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:14:25.958283 24355 external_mini_cluster.cc:1428] Setting key 11d4b1707124ec2aa8e6d841b6f39487
I20260807 20:14:25.959295 10397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355ae27b925f4710b0b37763bd90d164" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39715 } }
I20260807 20:14:25.959530 10397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:14:25.959646 10397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 355ae27b925f4710b0b37763bd90d164, State: Initialized, Role: FOLLOWER
I20260807 20:14:25.959803 10397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355ae27b925f4710b0b37763bd90d164" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39715 } }
I20260807 20:14:25.959877 10397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:14:25.959939 10397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:14:25.960011 10397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:14:25.960747 10397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355ae27b925f4710b0b37763bd90d164" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39715 } }
I20260807 20:14:25.960858 10397 leader_election.cc:302] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 355ae27b925f4710b0b37763bd90d164; no voters:
I20260807 20:14:25.961028 10397 leader_election.cc:288] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:14:25.961131 10404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:14:25.961387 10404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [term 1 LEADER]: Becoming Leader. State: Replica: 355ae27b925f4710b0b37763bd90d164, State: Running, Role: LEADER
I20260807 20:14:25.961395 10397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:14:25.961596 10404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355ae27b925f4710b0b37763bd90d164" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39715 } }
I20260807 20:14:25.961941 10404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "355ae27b925f4710b0b37763bd90d164" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355ae27b925f4710b0b37763bd90d164" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39715 } } }
I20260807 20:14:25.962019 10404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [sys.catalog]: This master's current role is: LEADER
I20260807 20:14:25.962163 10404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 355ae27b925f4710b0b37763bd90d164. Latest consensus state: current_term: 1 leader_uuid: "355ae27b925f4710b0b37763bd90d164" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355ae27b925f4710b0b37763bd90d164" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39715 } } }
I20260807 20:14:25.962230 10404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164 [sys.catalog]: This master's current role is: LEADER
I20260807 20:14:25.963148 10412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:14:25.963452 10397 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 20:14:25.963788 10412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 20:14:25 dist-test-slave-2j7r krb5kdc[9610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:14:25.971164 10412 catalog_manager.cc:1347] Generated new cluster ID: 5f87f99b7180445db749984fe419036e
I20260807 20:14:25.971316 10412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:14:25.973860 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:25.958874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55358 (local address 127.23.200.254:39715)
0807 20:14:25.959048 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 20:14:25.959058 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:14:25.959094 (+ 36us) server_negotiation.cc:407] Connection header received
0807 20:14:25.959276 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:14:25.959284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:14:25.959345 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:14:25.959507 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:14:25.960421 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:25.961870 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:25.962967 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:25.963568 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:25.964008 (+ 440us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:25.966272 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:14:25.966277 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:14:25.966277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:14:25.966291 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:14:25.971383 (+ 5092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:14:25.971868 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:14:25.971871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:14:25.971872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:14:25.971992 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:14:25.972174 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:14:25.972174 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:14:25.972174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:14:25.972564 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:14:25.973124 (+ 560us) server_negotiation.cc:1035] Waiting for connection context
0807 20:14:25.973194 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 20:14:25.973549 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
I20260807 20:14:25.981155 10412 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:14:25.982007 10412 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:14:25.988819 10412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164: Generated new TSK 0
I20260807 20:14:25.989032 10412 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:14:26.708415 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.200.212:52929/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39715
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:14:26.827486 10476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:14:26.827687 10476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:14:26.827721 10476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:14:26.828871 10476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:14:26.828908 10476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:14:26.828943 10476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:14:26.830470 10476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.23.200.212:52929/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:14:26.830701 10476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:14:26.830914 10476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:14:26.836938 10482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:14:26.836949 10487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:14:26.837162 10476 server_base.cc:1034] running on GCE node
W20260807 20:14:26.837317 10484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:14:26.837549 10476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:14:26.837802 10476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:14:26.838963 10476 hybrid_clock.cc:648] HybridClock initialized: now 1786133666838943 us; error 38 us; skew 500 ppm
Aug 07 20:14:26 dist-test-slave-2j7r krb5kdc[9610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133666, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:14:26.841616 10476 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:14:26.842079 10476 webserver.cc:460] Webserver started at http://127.23.200.193:42601/ using document root <none> and password file <none>
I20260807 20:14:26.842350 10476 fs_manager.cc:359] Metadata directory not provided
I20260807 20:14:26.842420 10476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:14:26.842607 10476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 20:14:26 dist-test-slave-2j7r krb5kdc[9610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133666, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
I20260807 20:14:26.868428 10476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "79a52edddcde4f828b64e76cd9e0e838"
format_stamp: "Formatted at 2026-08-07 20:14:26 on dist-test-slave-2j7r"
server_key: "4a429661d40a8a1568244f516c4f23ec"
server_key_iv: "d9d622db3216bf4baa6e0bb5ed2c47ee"
server_key_version: "kuduclusterkey@0"
I20260807 20:14:26.868907 10476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "79a52edddcde4f828b64e76cd9e0e838"
format_stamp: "Formatted at 2026-08-07 20:14:26 on dist-test-slave-2j7r"
server_key: "4a429661d40a8a1568244f516c4f23ec"
server_key_iv: "d9d622db3216bf4baa6e0bb5ed2c47ee"
server_key_version: "kuduclusterkey@0"
I20260807 20:14:26.871937 10476 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.001s
I20260807 20:14:26.886323 10500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:14:26.886569 10476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:14:26.886674 10476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "79a52edddcde4f828b64e76cd9e0e838"
format_stamp: "Formatted at 2026-08-07 20:14:26 on dist-test-slave-2j7r"
server_key: "4a429661d40a8a1568244f516c4f23ec"
server_key_iv: "d9d622db3216bf4baa6e0bb5ed2c47ee"
server_key_version: "kuduclusterkey@0"
I20260807 20:14:26.886766 10476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:14:26.911652 10476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:14:26.913694 10476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:14:26.913846 10476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:14:26.914108 10476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:14:26.914433 10476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:14:26.914470 10476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:14:26.914499 10476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:14:26.914522 10476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:14:26.923576 10476 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:44051
I20260807 20:14:26.923626 10689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:44051 every 8 connection(s)
I20260807 20:14:26.924091 10476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:14:26.927099 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 10476
I20260807 20:14:26.927209 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:14:26.927393 24355 external_mini_cluster.cc:1428] Setting key 6068bc4bfe20a03f420e657b466509c6
Aug 07 20:14:26 dist-test-slave-2j7r krb5kdc[9610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133666, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:14:26.945834 10698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:26.924902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39715 (local address 127.23.200.193:50311)
0807 20:14:26.925070 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 20:14:26.925134 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 20:14:26.925809 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:14:26.936002 (+ 10193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:14:26.936009 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:14:26.936303 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:14:26.936602 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:14:26.936610 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:14:26.937681 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:14:26.937682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:14:26.938996 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:14:26.938999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:14:26.939575 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:14:26.939576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:14:26.939719 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:26.940310 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:14:26.940321 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:14:26.942117 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:14:26.944122 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:14:26.944129 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:14:26.944132 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:14:26.944431 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:14:26.944657 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:14:26.944658 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:14:26.944659 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:14:26.944796 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:14:26.945141 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:14:26.945148 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:14:26.945317 (+ 169us) client_negotiation.cc:769] Sending connection context
0807 20:14:26.945423 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 20:14:26.945582 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260807 20:14:26.946470 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:26.925227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:50311 (local address 127.23.200.254:39715)
0807 20:14:26.935684 (+ 10457us) server_negotiation.cc:206] Beginning negotiation
0807 20:14:26.935687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:14:26.935700 (+ 13us) server_negotiation.cc:407] Connection header received
0807 20:14:26.935728 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:14:26.935729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:14:26.935741 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:14:26.935815 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:14:26.936719 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:26.937568 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:26.939093 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:26.939408 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:26.939467 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:26.942227 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:14:26.942228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:14:26.942229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:14:26.942237 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:14:26.944028 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:14:26.944518 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:14:26.944519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:14:26.944520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:14:26.944588 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:14:26.944868 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:14:26.944869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:14:26.944869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:14:26.945021 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:14:26.945114 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 20:14:26.946278 (+ 1164us) server_negotiation.cc:299] Negotiation successful
0807 20:14:26.946370 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10422,"thread_start_us":94,"threads_started":1}
I20260807 20:14:26.947348 10690 heartbeater.cc:347] Connected to a master server at 127.23.200.254:39715
I20260807 20:14:26.947448 10690 heartbeater.cc:464] Registering TS with master...
I20260807 20:14:26.947652 10690 heartbeater.cc:511] Master 127.23.200.254:39715 requested a full tablet report, sending...
I20260807 20:14:26.948225 10348 ts_manager.cc:194] Registered new tserver with Master: 79a52edddcde4f828b64e76cd9e0e838 (127.23.200.193:44051)
I20260807 20:14:26.949137 10348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:50311
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:14:26.982110 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.200.212:52929/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39715
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 20:14:27 dist-test-slave-2j7r krb5kdc[9610](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786133667, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 20:14:27.105293 10706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:14:27.105640 10706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:14:27.105697 10706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:14:27.107419 10706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:14:27.107492 10706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:14:27.107532 10706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:14:27.109722 10706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.23.200.212:52929/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:14:27.109977 10706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:14:27.110234 10706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:14:27.115334 10717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:14:27.115474 10721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:14:27.115334 10715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:14:27.115796 10706 server_base.cc:1034] running on GCE node
I20260807 20:14:27.115978 10706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:14:27.116242 10706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:14:27.117400 10706 hybrid_clock.cc:648] HybridClock initialized: now 1786133667117376 us; error 39 us; skew 500 ppm
Aug 07 20:14:27 dist-test-slave-2j7r krb5kdc[9610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133667, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:14:27.120710 10706 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:14:27.121389 10706 webserver.cc:460] Webserver started at http://127.23.200.194:38191/ using document root <none> and password file <none>
I20260807 20:14:27.121654 10706 fs_manager.cc:359] Metadata directory not provided
I20260807 20:14:27.121717 10706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:14:27.121850 10706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 20:14:27 dist-test-slave-2j7r krb5kdc[9610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133667, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
I20260807 20:14:27.136951 10706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "cbfae6b44800409e981c7c00d3e039ae"
format_stamp: "Formatted at 2026-08-07 20:14:27 on dist-test-slave-2j7r"
server_key: "6e1141d6b7d62e44494bd9c33679cd7c"
server_key_iv: "ae4d4f9dc4d990514096aa06251ae5cd"
server_key_version: "kuduclusterkey@0"
I20260807 20:14:27.137588 10706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "cbfae6b44800409e981c7c00d3e039ae"
format_stamp: "Formatted at 2026-08-07 20:14:27 on dist-test-slave-2j7r"
server_key: "6e1141d6b7d62e44494bd9c33679cd7c"
server_key_iv: "ae4d4f9dc4d990514096aa06251ae5cd"
server_key_version: "kuduclusterkey@0"
I20260807 20:14:27.141530 10706 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260807 20:14:27.173271 10731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:14:27.173583 10706 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.001s sys 0.000s
I20260807 20:14:27.173718 10706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "cbfae6b44800409e981c7c00d3e039ae"
format_stamp: "Formatted at 2026-08-07 20:14:27 on dist-test-slave-2j7r"
server_key: "6e1141d6b7d62e44494bd9c33679cd7c"
server_key_iv: "ae4d4f9dc4d990514096aa06251ae5cd"
server_key_version: "kuduclusterkey@0"
I20260807 20:14:27.173834 10706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:14:27.219980 10706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:14:27.222771 10706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:14:27.223031 10706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:14:27.223353 10706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:14:27.223837 10706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:14:27.223948 10706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:14:27.224026 10706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:14:27.224085 10706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:14:27.232378 10706 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:35015
I20260807 20:14:27.233240 10706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:14:27.236186 10907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:35015 every 8 connection(s)
I20260807 20:14:27.238142 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 10706
I20260807 20:14:27.238251 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:14:27.238458 24355 external_mini_cluster.cc:1428] Setting key 443b6bfc9dfc046e6361f3e91c53e756
Aug 07 20:14:27 dist-test-slave-2j7r krb5kdc[9610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133667, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:14:27.251946 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.237315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:43285 (local address 127.23.200.254:39715)
0807 20:14:27.237364 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:14:27.237366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:14:27.237731 (+ 365us) server_negotiation.cc:407] Connection header received
0807 20:14:27.241935 (+ 4204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:14:27.241938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:14:27.241956 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:14:27.241989 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:14:27.242974 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.243766 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.245375 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.245699 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.245746 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:27.248453 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:14:27.248455 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:14:27.248455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:14:27.248463 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:14:27.250174 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:14:27.250680 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:14:27.250682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:14:27.250682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:14:27.250742 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:14:27.251072 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:14:27.251073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:14:27.251074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:14:27.251243 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:14:27.251318 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 20:14:27.251720 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 20:14:27.251823 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:14:27.252895 10910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.236959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39715 (local address 127.23.200.194:43285)
0807 20:14:27.237589 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0807 20:14:27.237638 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 20:14:27.241781 (+ 4143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:14:27.242254 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:14:27.242260 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:14:27.242530 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:14:27.242837 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:14:27.242844 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.243952 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.243953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:14:27.245269 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:14:27.245273 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.245847 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.245847 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:14:27.246034 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:27.246633 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:14:27.246644 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:14:27.248341 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:14:27.250272 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:14:27.250279 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:14:27.250282 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:14:27.250588 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:14:27.250816 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:14:27.250817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:14:27.250817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:14:27.250984 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:14:27.251371 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:14:27.251378 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:14:27.251552 (+ 174us) client_negotiation.cc:769] Sending connection context
0807 20:14:27.252473 (+ 921us) client_negotiation.cc:240] Negotiation successful
0807 20:14:27.252651 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":97,"threads_started":1}
I20260807 20:14:27.253628 10908 heartbeater.cc:347] Connected to a master server at 127.23.200.254:39715
I20260807 20:14:27.253728 10908 heartbeater.cc:464] Registering TS with master...
I20260807 20:14:27.253945 10908 heartbeater.cc:511] Master 127.23.200.254:39715 requested a full tablet report, sending...
I20260807 20:14:27.254390 10348 ts_manager.cc:194] Registered new tserver with Master: cbfae6b44800409e981c7c00d3e039ae (127.23.200.194:35015)
I20260807 20:14:27.254979 10348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:43285
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:14:27.287868 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.200.212:52929/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39715
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:14:27.412297 10913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:14:27.412628 10913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:14:27.412707 10913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:14:27.414433 10913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:14:27.414634 10913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:14:27.414736 10913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:14:27.417011 10913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.23.200.212:52929/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:14:27.417505 10913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:14:27.417829 10913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:14:27.424407 10921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:14:27.425004 10920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:14:27.425307 10923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:14:27.425509 10913 server_base.cc:1034] running on GCE node
I20260807 20:14:27.425860 10913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:14:27.426126 10913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:14:27.427843 10913 hybrid_clock.cc:648] HybridClock initialized: now 1786133667427816 us; error 38 us; skew 500 ppm
Aug 07 20:14:27 dist-test-slave-2j7r krb5kdc[9610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133667, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:14:27.431082 10913 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:14:27.431556 10913 webserver.cc:460] Webserver started at http://127.23.200.195:34787/ using document root <none> and password file <none>
I20260807 20:14:27.431825 10913 fs_manager.cc:359] Metadata directory not provided
I20260807 20:14:27.431871 10913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:14:27.431989 10913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 20:14:27 dist-test-slave-2j7r krb5kdc[9610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133667, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
I20260807 20:14:27.446249 10913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "e4e8518dce8d48bbae80be5f4a7d2cc8"
format_stamp: "Formatted at 2026-08-07 20:14:27 on dist-test-slave-2j7r"
server_key: "20bda7259d3a17791211388a9b37d8af"
server_key_iv: "a2ac3350fb330365daf1dead78d3e20b"
server_key_version: "kuduclusterkey@0"
I20260807 20:14:27.446900 10913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "e4e8518dce8d48bbae80be5f4a7d2cc8"
format_stamp: "Formatted at 2026-08-07 20:14:27 on dist-test-slave-2j7r"
server_key: "20bda7259d3a17791211388a9b37d8af"
server_key_iv: "a2ac3350fb330365daf1dead78d3e20b"
server_key_version: "kuduclusterkey@0"
I20260807 20:14:27.449725 10913 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260807 20:14:27.464334 10931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:14:27.464593 10913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:14:27.464684 10913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "e4e8518dce8d48bbae80be5f4a7d2cc8"
format_stamp: "Formatted at 2026-08-07 20:14:27 on dist-test-slave-2j7r"
server_key: "20bda7259d3a17791211388a9b37d8af"
server_key_iv: "a2ac3350fb330365daf1dead78d3e20b"
server_key_version: "kuduclusterkey@0"
I20260807 20:14:27.464777 10913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:14:27.486542 10913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:14:27.489066 10913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:14:27.489286 10913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:14:27.489537 10913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:14:27.489948 10913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:14:27.490031 10913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:14:27.490097 10913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:14:27.490149 10913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:14:27.496644 10913 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:46733
I20260807 20:14:27.497182 10913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:14:27.500171 11093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:46733 every 8 connection(s)
I20260807 20:14:27.501307 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 10913
I20260807 20:14:27.501408 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:14:27.501600 24355 external_mini_cluster.cc:1428] Setting key 0a978d0fb7103d53383b12a0b11df285
Aug 07 20:14:27 dist-test-slave-2j7r krb5kdc[9610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133667, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:14:27.511958 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.501228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:54803 (local address 127.23.200.254:39715)
0807 20:14:27.501273 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 20:14:27.501275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:14:27.501615 (+ 340us) server_negotiation.cc:407] Connection header received
0807 20:14:27.502547 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:14:27.502548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:14:27.502559 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:14:27.502585 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:14:27.503433 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.504210 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.505647 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.505942 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.506002 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:27.508545 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:14:27.508547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:14:27.508548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:14:27.508555 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:14:27.510204 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:14:27.510949 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:14:27.510951 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:14:27.510951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:14:27.511011 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:14:27.511261 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:14:27.511262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:14:27.511263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:14:27.511387 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:14:27.511426 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 20:14:27.511764 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 20:14:27.511857 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:14:27.512614 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.500895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39715 (local address 127.23.200.195:54803)
0807 20:14:27.501482 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0807 20:14:27.501528 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 20:14:27.502431 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:14:27.502767 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:14:27.502771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:14:27.503050 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:14:27.503328 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:14:27.503335 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.504318 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.504319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:14:27.505554 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:14:27.505557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.506090 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.506090 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:14:27.506226 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:27.506825 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:14:27.506836 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:14:27.508439 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:14:27.510295 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:14:27.510303 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:14:27.510305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:14:27.510856 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:14:27.511073 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:14:27.511074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:14:27.511074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:14:27.511198 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:14:27.511467 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:14:27.511473 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:14:27.511628 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 20:14:27.512215 (+ 587us) client_negotiation.cc:240] Negotiation successful
0807 20:14:27.512377 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":91,"threads_started":1}
I20260807 20:14:27.513497 11095 heartbeater.cc:347] Connected to a master server at 127.23.200.254:39715
I20260807 20:14:27.513602 11095 heartbeater.cc:464] Registering TS with master...
I20260807 20:14:27.513792 11095 heartbeater.cc:511] Master 127.23.200.254:39715 requested a full tablet report, sending...
I20260807 20:14:27.514143 10347 ts_manager.cc:194] Registered new tserver with Master: e4e8518dce8d48bbae80be5f4a7d2cc8 (127.23.200.195:46733)
I20260807 20:14:27.514614 10347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:54803
I20260807 20:14:27.524343 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:14:27.533674 10699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.526264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51964 (local address 127.23.200.254:39715)
0807 20:14:27.526307 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:14:27.526309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:14:27.526771 (+ 462us) server_negotiation.cc:407] Connection header received
0807 20:14:27.526879 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:14:27.526881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:14:27.526893 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:14:27.526922 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:14:27.527666 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.528362 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.529396 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.529650 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.529687 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:27.531051 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:14:27.531053 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:14:27.531053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:14:27.531061 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:14:27.532496 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:14:27.532881 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:14:27.532882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:14:27.532882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:14:27.532939 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:14:27.533137 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:14:27.533138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:14:27.533139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:14:27.533255 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:14:27.533322 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 20:14:27.533511 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 20:14:27.533583 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 07 20:14:27 dist-test-slave-2j7r krb5kdc[9610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133667, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
I20260807 20:14:27.535959 10347 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:14:27.538141 10347 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:14:27.547462 11121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.541254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:46733 (local address 127.0.0.1:40114)
0807 20:14:27.542332 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0807 20:14:27.542373 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:14:27.542430 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:14:27.542948 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:14:27.542951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:14:27.542953 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:14:27.543016 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:14:27.543017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.544506 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.544507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:14:27.545775 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:14:27.545778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.547174 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.547175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:14:27.547253 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:27.547266 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 20:14:27.547329 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 20:14:27.547381 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1046,"thread_start_us":71,"threads_started":1}
I20260807 20:14:27.548030 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.541521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40114 (local address 127.23.200.195:46733)
0807 20:14:27.542673 (+ 1152us) server_negotiation.cc:206] Beginning negotiation
0807 20:14:27.542678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:14:27.542688 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:14:27.542727 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:14:27.542729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:14:27.542757 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:14:27.542845 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:14:27.543085 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.544430 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.545925 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.546986 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.547070 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:27.547140 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:14:27.547897 (+ 757us) server_negotiation.cc:299] Negotiation successful
0807 20:14:27.547940 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1114,"thread_start_us":74,"threads_started":1}
I20260807 20:14:27.548835 11001 tablet_service.cc:1467] Processing CreateTablet for tablet ce69d7bc30d54f5ea77af36b730e1548 (DEFAULT_TABLE table=test-table [id=ac758b0b70b84dce80f7668d61af6f90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:14:27.549257 11001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce69d7bc30d54f5ea77af36b730e1548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:14:27.552513 11127 tablet_bootstrap.cc:492] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8: Bootstrap starting.
I20260807 20:14:27.553553 11127 tablet_bootstrap.cc:654] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8: Neither blocks nor log segments found. Creating new log.
I20260807 20:14:27.553901 11127 log.cc:824] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8: Log is configured to *not* fsync() on all Append() calls
I20260807 20:14:27.563612 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.541759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:35015 (local address 127.0.0.1:42912)
0807 20:14:27.542522 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0807 20:14:27.542529 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 20:14:27.542559 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:14:27.549583 (+ 7024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:14:27.549584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:14:27.549587 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:14:27.549629 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:14:27.549631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.559746 (+ 10115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.559748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:14:27.560882 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:14:27.560886 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.563368 (+ 2482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.563369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:14:27.563436 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:27.563441 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:14:27.563485 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 20:14:27.563532 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":89,"threads_started":1}
I20260807 20:14:27.564123 11122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.541889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42912 (local address 127.23.200.194:35015)
0807 20:14:27.549203 (+ 7314us) server_negotiation.cc:206] Beginning negotiation
0807 20:14:27.549209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:14:27.549223 (+ 14us) server_negotiation.cc:407] Connection header received
0807 20:14:27.549265 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:14:27.549268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:14:27.549286 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:14:27.549485 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:14:27.549711 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.551081 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.562128 (+ 11047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.563175 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.563260 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:27.563333 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 20:14:27.563999 (+ 666us) server_negotiation.cc:299] Negotiation successful
0807 20:14:27.564043 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7265,"thread_start_us":86,"threads_started":1}
I20260807 20:14:27.564949 10801 tablet_service.cc:1467] Processing CreateTablet for tablet ce69d7bc30d54f5ea77af36b730e1548 (DEFAULT_TABLE table=test-table [id=ac758b0b70b84dce80f7668d61af6f90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:14:27.565364 10801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce69d7bc30d54f5ea77af36b730e1548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:14:27.567281 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.541928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:44051 (local address 127.0.0.1:51762)
0807 20:14:27.545152 (+ 3224us) negotiation.cc:107] Waiting for socket to connect
0807 20:14:27.545163 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:14:27.545216 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:14:27.557457 (+ 12241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:14:27.557458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:14:27.557462 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:14:27.557512 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:14:27.557514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.562370 (+ 4856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.562371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:14:27.563388 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:14:27.563393 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.567060 (+ 3667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.567062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:14:27.567131 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:27.567137 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 20:14:27.567164 (+ 27us) client_negotiation.cc:240] Negotiation successful
0807 20:14:27.567207 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3205,"thread_start_us":64,"threads_started":1}
I20260807 20:14:27.567672 11126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.544700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51762 (local address 127.23.200.193:44051)
0807 20:14:27.557295 (+ 12595us) server_negotiation.cc:206] Beginning negotiation
0807 20:14:27.557300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:14:27.557311 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:14:27.557348 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:14:27.557350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:14:27.557368 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:14:27.557455 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:14:27.557612 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.558682 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.564491 (+ 5809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.566960 (+ 2469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.567449 (+ 489us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:27.567533 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 20:14:27.567560 (+ 27us) server_negotiation.cc:299] Negotiation successful
0807 20:14:27.567603 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12553,"thread_start_us":55,"threads_started":1}
I20260807 20:14:27.568368 10564 tablet_service.cc:1467] Processing CreateTablet for tablet ce69d7bc30d54f5ea77af36b730e1548 (DEFAULT_TABLE table=test-table [id=ac758b0b70b84dce80f7668d61af6f90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:14:27.568712 10564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce69d7bc30d54f5ea77af36b730e1548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:14:27.569420 11127 tablet_bootstrap.cc:492] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8: No bootstrap required, opened a new log
I20260807 20:14:27.569490 11127 ts_tablet_manager.cc:1397] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20260807 20:14:27.571547 11127 raft_consensus.cc:348] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4e8518dce8d48bbae80be5f4a7d2cc8" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46733 } } peers { permanent_uuid: "cbfae6b44800409e981c7c00d3e039ae" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35015 } } peers { permanent_uuid: "79a52edddcde4f828b64e76cd9e0e838" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44051 } }
I20260807 20:14:27.571717 11127 raft_consensus.cc:374] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:14:27.571746 11127 raft_consensus.cc:729] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4e8518dce8d48bbae80be5f4a7d2cc8, State: Initialized, Role: FOLLOWER
I20260807 20:14:27.571893 11127 consensus_queue.cc:260] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4e8518dce8d48bbae80be5f4a7d2cc8" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46733 } } peers { permanent_uuid: "cbfae6b44800409e981c7c00d3e039ae" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35015 } } peers { permanent_uuid: "79a52edddcde4f828b64e76cd9e0e838" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44051 } }
I20260807 20:14:27.572171 11127 ts_tablet_manager.cc:1428] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:14:27.572818 11095 heartbeater.cc:503] Master 127.23.200.254:39715 was elected leader, sending a full tablet report...
I20260807 20:14:27.574820 11129 tablet_bootstrap.cc:492] T ce69d7bc30d54f5ea77af36b730e1548 P 79a52edddcde4f828b64e76cd9e0e838: Bootstrap starting.
I20260807 20:14:27.575572 11129 tablet_bootstrap.cc:654] T ce69d7bc30d54f5ea77af36b730e1548 P 79a52edddcde4f828b64e76cd9e0e838: Neither blocks nor log segments found. Creating new log.
I20260807 20:14:27.575870 11129 log.cc:824] T ce69d7bc30d54f5ea77af36b730e1548 P 79a52edddcde4f828b64e76cd9e0e838: Log is configured to *not* fsync() on all Append() calls
I20260807 20:14:27.576220 11134 tablet_bootstrap.cc:492] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae: Bootstrap starting.
I20260807 20:14:27.577270 11134 tablet_bootstrap.cc:654] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae: Neither blocks nor log segments found. Creating new log.
I20260807 20:14:27.577564 11134 log.cc:824] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae: Log is configured to *not* fsync() on all Append() calls
I20260807 20:14:27.579730 11134 tablet_bootstrap.cc:492] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae: No bootstrap required, opened a new log
I20260807 20:14:27.579797 11134 ts_tablet_manager.cc:1397] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 20:14:27.581991 11134 raft_consensus.cc:348] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4e8518dce8d48bbae80be5f4a7d2cc8" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46733 } } peers { permanent_uuid: "cbfae6b44800409e981c7c00d3e039ae" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35015 } } peers { permanent_uuid: "79a52edddcde4f828b64e76cd9e0e838" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44051 } }
I20260807 20:14:27.582170 11134 raft_consensus.cc:374] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:14:27.582201 11134 raft_consensus.cc:729] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbfae6b44800409e981c7c00d3e039ae, State: Initialized, Role: FOLLOWER
I20260807 20:14:27.582309 11134 consensus_queue.cc:260] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4e8518dce8d48bbae80be5f4a7d2cc8" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46733 } } peers { permanent_uuid: "cbfae6b44800409e981c7c00d3e039ae" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35015 } } peers { permanent_uuid: "79a52edddcde4f828b64e76cd9e0e838" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44051 } }
I20260807 20:14:27.582582 11134 ts_tablet_manager.cc:1428] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:14:27.582713 10908 heartbeater.cc:503] Master 127.23.200.254:39715 was elected leader, sending a full tablet report...
I20260807 20:14:27.583735 11129 tablet_bootstrap.cc:492] T ce69d7bc30d54f5ea77af36b730e1548 P 79a52edddcde4f828b64e76cd9e0e838: No bootstrap required, opened a new log
I20260807 20:14:27.583820 11129 ts_tablet_manager.cc:1397] T ce69d7bc30d54f5ea77af36b730e1548 P 79a52edddcde4f828b64e76cd9e0e838: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260807 20:14:27.586385 11129 raft_consensus.cc:348] T ce69d7bc30d54f5ea77af36b730e1548 P 79a52edddcde4f828b64e76cd9e0e838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4e8518dce8d48bbae80be5f4a7d2cc8" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46733 } } peers { permanent_uuid: "cbfae6b44800409e981c7c00d3e039ae" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35015 } } peers { permanent_uuid: "79a52edddcde4f828b64e76cd9e0e838" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44051 } }
I20260807 20:14:27.586565 11129 raft_consensus.cc:374] T ce69d7bc30d54f5ea77af36b730e1548 P 79a52edddcde4f828b64e76cd9e0e838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:14:27.589900 11129 raft_consensus.cc:729] T ce69d7bc30d54f5ea77af36b730e1548 P 79a52edddcde4f828b64e76cd9e0e838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79a52edddcde4f828b64e76cd9e0e838, State: Initialized, Role: FOLLOWER
I20260807 20:14:27.590531 11129 consensus_queue.cc:260] T ce69d7bc30d54f5ea77af36b730e1548 P 79a52edddcde4f828b64e76cd9e0e838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4e8518dce8d48bbae80be5f4a7d2cc8" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46733 } } peers { permanent_uuid: "cbfae6b44800409e981c7c00d3e039ae" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35015 } } peers { permanent_uuid: "79a52edddcde4f828b64e76cd9e0e838" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44051 } }
I20260807 20:14:27.590873 11129 ts_tablet_manager.cc:1428] T ce69d7bc30d54f5ea77af36b730e1548 P 79a52edddcde4f828b64e76cd9e0e838: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260807 20:14:27.591017 10690 heartbeater.cc:503] Master 127.23.200.254:39715 was elected leader, sending a full tablet report...
I20260807 20:14:27.650084 11137 raft_consensus.cc:482] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:14:27.650238 11137 raft_consensus.cc:504] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4e8518dce8d48bbae80be5f4a7d2cc8" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46733 } } peers { permanent_uuid: "cbfae6b44800409e981c7c00d3e039ae" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35015 } } peers { permanent_uuid: "79a52edddcde4f828b64e76cd9e0e838" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44051 } }
I20260807 20:14:27.650624 11137 leader_election.cc:288] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4e8518dce8d48bbae80be5f4a7d2cc8 (127.23.200.195:46733), 79a52edddcde4f828b64e76cd9e0e838 (127.23.200.193:44051)
I20260807 20:14:27.654959 10910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.650856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:44051 (local address 127.23.200.194:55959)
0807 20:14:27.651037 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 20:14:27.651048 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:14:27.651091 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:14:27.651841 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:14:27.651842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:14:27.651844 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:14:27.651922 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:14:27.651923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.652793 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.652794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:14:27.653955 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:14:27.653958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.654716 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.654717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:14:27.654790 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:27.654793 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:14:27.654847 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 20:14:27.654885 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260807 20:14:27.655436 11126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.650994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:55959 (local address 127.23.200.193:44051)
0807 20:14:27.651457 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0807 20:14:27.651459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:14:27.651468 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:14:27.651496 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:14:27.651497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:14:27.651508 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:14:27.651555 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:14:27.652000 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.652694 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.654079 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.654614 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.654678 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:27.654687 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:14:27.655329 (+ 642us) server_negotiation.cc:299] Negotiation successful
0807 20:14:27.655372 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436}
I20260807 20:14:27.655699 10584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce69d7bc30d54f5ea77af36b730e1548" candidate_uuid: "cbfae6b44800409e981c7c00d3e039ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79a52edddcde4f828b64e76cd9e0e838" is_pre_election: true
I20260807 20:14:27.655869 10584 raft_consensus.cc:2454] T ce69d7bc30d54f5ea77af36b730e1548 P 79a52edddcde4f828b64e76cd9e0e838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbfae6b44800409e981c7c00d3e039ae in term 0.
I20260807 20:14:27.656157 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.651751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:52035 (local address 127.23.200.195:46733)
0807 20:14:27.651793 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:14:27.651794 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:14:27.652883 (+ 1089us) server_negotiation.cc:407] Connection header received
0807 20:14:27.652994 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:14:27.652995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:14:27.653006 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:14:27.653035 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:14:27.653288 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.654004 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.655315 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.655813 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.655858 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:27.655865 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 20:14:27.656049 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 20:14:27.656089 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:14:27.656185 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.651224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:46733 (local address 127.23.200.194:52035)
0807 20:14:27.652787 (+ 1563us) negotiation.cc:107] Waiting for socket to connect
0807 20:14:27.652802 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 20:14:27.652917 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:14:27.653120 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:14:27.653121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:14:27.653123 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:14:27.653204 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:14:27.653205 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.654128 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.654129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:14:27.655202 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:14:27.655205 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.655923 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.655923 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:14:27.655988 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:27.655992 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:14:27.656042 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 20:14:27.656079 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1537,"thread_start_us":126,"threads_started":1}
I20260807 20:14:27.656152 10732 leader_election.cc:302] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79a52edddcde4f828b64e76cd9e0e838, cbfae6b44800409e981c7c00d3e039ae; no voters:
I20260807 20:14:27.656345 11137 raft_consensus.cc:2781] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:14:27.656392 11137 raft_consensus.cc:482] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:14:27.656411 11137 raft_consensus.cc:3037] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:14:27.657310 11137 raft_consensus.cc:504] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4e8518dce8d48bbae80be5f4a7d2cc8" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46733 } } peers { permanent_uuid: "cbfae6b44800409e981c7c00d3e039ae" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35015 } } peers { permanent_uuid: "79a52edddcde4f828b64e76cd9e0e838" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44051 } }
I20260807 20:14:27.657444 11137 leader_election.cc:288] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [CANDIDATE]: Term 1 election: Requested vote from peers e4e8518dce8d48bbae80be5f4a7d2cc8 (127.23.200.195:46733), 79a52edddcde4f828b64e76cd9e0e838 (127.23.200.193:44051)
I20260807 20:14:27.657668 10584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce69d7bc30d54f5ea77af36b730e1548" candidate_uuid: "cbfae6b44800409e981c7c00d3e039ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79a52edddcde4f828b64e76cd9e0e838"
I20260807 20:14:27.657753 10584 raft_consensus.cc:3037] T ce69d7bc30d54f5ea77af36b730e1548 P 79a52edddcde4f828b64e76cd9e0e838 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:14:27.657861 11027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce69d7bc30d54f5ea77af36b730e1548" candidate_uuid: "cbfae6b44800409e981c7c00d3e039ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4e8518dce8d48bbae80be5f4a7d2cc8" is_pre_election: true
I20260807 20:14:27.657872 11024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce69d7bc30d54f5ea77af36b730e1548" candidate_uuid: "cbfae6b44800409e981c7c00d3e039ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4e8518dce8d48bbae80be5f4a7d2cc8"
I20260807 20:14:27.658049 11027 raft_consensus.cc:2454] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbfae6b44800409e981c7c00d3e039ae in term 0.
I20260807 20:14:27.658794 10584 raft_consensus.cc:2454] T ce69d7bc30d54f5ea77af36b730e1548 P 79a52edddcde4f828b64e76cd9e0e838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbfae6b44800409e981c7c00d3e039ae in term 1.
I20260807 20:14:27.658972 10732 leader_election.cc:302] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79a52edddcde4f828b64e76cd9e0e838, cbfae6b44800409e981c7c00d3e039ae; no voters: e4e8518dce8d48bbae80be5f4a7d2cc8
I20260807 20:14:27.659082 11137 raft_consensus.cc:2781] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:14:27.659235 11137 raft_consensus.cc:686] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [term 1 LEADER]: Becoming Leader. State: Replica: cbfae6b44800409e981c7c00d3e039ae, State: Running, Role: LEADER
I20260807 20:14:27.659346 11137 consensus_queue.cc:237] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4e8518dce8d48bbae80be5f4a7d2cc8" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46733 } } peers { permanent_uuid: "cbfae6b44800409e981c7c00d3e039ae" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35015 } } peers { permanent_uuid: "79a52edddcde4f828b64e76cd9e0e838" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44051 } }
I20260807 20:14:27.660192 10347 catalog_manager.cc:5495] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae reported cstate change: term changed from 0 to 1, leader changed from <none> to cbfae6b44800409e981c7c00d3e039ae (127.23.200.194). New cstate: current_term: 1 leader_uuid: "cbfae6b44800409e981c7c00d3e039ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4e8518dce8d48bbae80be5f4a7d2cc8" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbfae6b44800409e981c7c00d3e039ae" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 35015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79a52edddcde4f828b64e76cd9e0e838" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44051 } health_report { overall_health: UNKNOWN } } }
I20260807 20:14:27.672061 11122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.669003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42914 (local address 127.23.200.194:35015)
0807 20:14:27.669051 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 20:14:27.669054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:14:27.669063 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:14:27.669132 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:14:27.669133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:14:27.669155 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:14:27.669234 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:14:27.669367 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.670086 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.671090 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:14:27.671411 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:14:27.671494 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:14:27.671686 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:14:27.671833 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:14:27.671927 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 20:14:27.671947 (+ 20us) server_negotiation.cc:299] Negotiation successful
0807 20:14:27.671974 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:14:27.674250 10584 raft_consensus.cc:1261] T ce69d7bc30d54f5ea77af36b730e1548 P 79a52edddcde4f828b64e76cd9e0e838 [term 1 FOLLOWER]: Refusing update from remote peer cbfae6b44800409e981c7c00d3e039ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:14:27.674253 11024 raft_consensus.cc:3037] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:14:27.675289 11024 raft_consensus.cc:1261] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8 [term 1 FOLLOWER]: Refusing update from remote peer cbfae6b44800409e981c7c00d3e039ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:14:27.675643 11140 consensus_queue.cc:1035] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "79a52edddcde4f828b64e76cd9e0e838" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:14:27.675871 11137 consensus_queue.cc:1035] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4e8518dce8d48bbae80be5f4a7d2cc8" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 46733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 20:14:27.677644 10693 tablet.cc:2369] T ce69d7bc30d54f5ea77af36b730e1548 P 79a52edddcde4f828b64e76cd9e0e838: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:14:27.682318 11146 mvcc.cc:205] Tried to move back new op lower bound from 7316003502790909952 to 7316003502733336576. Current Snapshot: MvccSnapshot[applied={T|T < 7316003502790909952}]
I20260807 20:14:27.683951 11148 mvcc.cc:205] Tried to move back new op lower bound from 7316003502790909952 to 7316003502733336576. Current Snapshot: MvccSnapshot[applied={T|T < 7316003502790909952}]
I20260807 20:14:27.684070 11147 mvcc.cc:205] Tried to move back new op lower bound from 7316003502790909952 to 7316003502733336576. Current Snapshot: MvccSnapshot[applied={T|T < 7316003502790909952 or (T in {7316003502790909952})}]
I20260807 20:14:27.686631 10347 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:14:27.686729 10347 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:14:27.687430 10347 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 355ae27b925f4710b0b37763bd90d164: Sending DeleteTablet for 3 replicas of tablet ce69d7bc30d54f5ea77af36b730e1548
I20260807 20:14:27.687767 11001 tablet_service.cc:1514] Processing DeleteTablet for tablet ce69d7bc30d54f5ea77af36b730e1548 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:14:27 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:40114
I20260807 20:14:27.688177 10801 tablet_service.cc:1514] Processing DeleteTablet for tablet ce69d7bc30d54f5ea77af36b730e1548 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:14:27 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:42912
I20260807 20:14:27.688405 10564 tablet_service.cc:1514] Processing DeleteTablet for tablet ce69d7bc30d54f5ea77af36b730e1548 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:14:27 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:51762
I20260807 20:14:27.688935 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 10476
I20260807 20:14:27.702194 11162 tablet_replica.cc:330] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae: stopping tablet replica
I20260807 20:14:27.702381 11162 raft_consensus.cc:2229] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:14:27.702531 11162 raft_consensus.cc:2258] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:14:27.708333 11162 ts_tablet_manager.cc:1905] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:14:27.710261 11161 tablet_replica.cc:330] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8: stopping tablet replica
I20260807 20:14:27.710445 11161 raft_consensus.cc:2229] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:14:27.710654 11162 ts_tablet_manager.cc:1918] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:14:27.710856 11161 raft_consensus.cc:2258] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:14:27.710973 11162 log.cc:1196] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/ce69d7bc30d54f5ea77af36b730e1548
I20260807 20:14:27.711422 11162 ts_tablet_manager.cc:1939] T ce69d7bc30d54f5ea77af36b730e1548 P cbfae6b44800409e981c7c00d3e039ae: Deleting consensus metadata
I20260807 20:14:27.713325 10326 catalog_manager.cc:4841] TS cbfae6b44800409e981c7c00d3e039ae (127.23.200.194:35015): tablet ce69d7bc30d54f5ea77af36b730e1548 (table test-table [id=ac758b0b70b84dce80f7668d61af6f90]) successfully deleted
W20260807 20:14:27.717535 10326 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:44051 (error 108)
I20260807 20:14:27.719195 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 10706
I20260807 20:14:27.719308 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.719064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:44051 (local address 127.0.0.1:51764)
0807 20:14:27.719210 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 20:14:27.719272 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260807 20:14:27.727712 11161 ts_tablet_manager.cc:1905] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:14:27.729631 11161 ts_tablet_manager.cc:1918] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:14:27.729715 11161 log.cc:1196] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/ce69d7bc30d54f5ea77af36b730e1548
I20260807 20:14:27.730300 11161 ts_tablet_manager.cc:1939] T ce69d7bc30d54f5ea77af36b730e1548 P e4e8518dce8d48bbae80be5f4a7d2cc8: Deleting consensus metadata
I20260807 20:14:27.730886 10328 catalog_manager.cc:4841] TS e4e8518dce8d48bbae80be5f4a7d2cc8 (127.23.200.195:46733): tablet ce69d7bc30d54f5ea77af36b730e1548 (table test-table [id=ac758b0b70b84dce80f7668d61af6f90]) successfully deleted
W20260807 20:14:27.758912 10326 catalog_manager.cc:4584] TS 79a52edddcde4f828b64e76cd9e0e838 (127.23.200.193:44051): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ce69d7bc30d54f5ea77af36b730e1548: Network error: Client connection negotiation failed: client connection to 127.23.200.193:44051: connect: Connection refused (error 111)
I20260807 20:14:27.807353 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 10913
I20260807 20:14:27.812486 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.812339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:44051 (local address 127.0.0.1:51774)
0807 20:14:27.812405 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 20:14:27.812439 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 20:14:27.812963 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:14:27.812844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:44051 (local address 127.0.0.1:51784)
0807 20:14:27.812884 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 20:14:27.812921 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 20:14:27.818401 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 10302
I20260807 20:14:27.830760 24355 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 20:14:28.175351 24355 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 20:14:28.175467 24355 mini_ranger.cc:66] Stopping Ranger...
I20260807 20:14:28.519809 24355 mini_ranger.cc:68] Stopped Ranger
2026-08-07 20:14:28.520 UTC [9655] LOG: received smart shutdown request
2026-08-07 20:14:28.523 UTC [9655] LOG: background worker "logical replication launcher" (PID 9674) exited with exit code 1
2026-08-07 20:14:28.524 UTC [9663] LOG: shutting down
2026-08-07 20:14:28.682 UTC [9655] LOG: database system is shut down
2026-08-07T20:14:28Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47535 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:14:28 dist-test-slave-2j7r krb5kdc[11177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:14:28 dist-test-slave-2j7r krb5kdc[11177](info): set up 2 sockets
Aug 07 20:14:28 dist-test-slave-2j7r krb5kdc[11177](info): commencing operation
krb5kdc: starting...
W20260807 20:14:29.969673 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:14:30 dist-test-slave-2j7r krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:14:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:14:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.23.200.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.23.200.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.23.200.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/rangeradmin_127.23.200.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.23.200.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/rangeradmin_127.23.200.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.23.200.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.23.200.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.23.200.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/rangerlookup_127.23.200.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.23.200.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/rangerlookup_127.23.200.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.200.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.23.200.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.200.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/HTTP_127.23.200.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.200.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/HTTP_127.23.200.212@KRBTEST.COM.keytab.
I20260807 20:14:30.143596 24355 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskISIRft/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/postgres -l logfile start
2026-08-07 20:14:32.571 UTC [11221] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 20:14:32.571 UTC [11221] LOG: listening on IPv4 address "127.23.200.212", port 59325
2026-08-07 20:14:32.575 UTC [11221] LOG: listening on Unix socket "/tmp/.s.PGSQL.59325"
2026-08-07 20:14:32.596 UTC [11225] LOG: database system was shut down at 2026-08-07 20:14:31 UTC
2026-08-07 20:14:32.599 UTC [11221] LOG: database system is ready to accept connections
I20260807 20:14:33.666252 24355 mini_postgres.cc:96] Postgres bound to 59325
2026-08-07 20:14:33.673 UTC [11237] FATAL: database "slave" does not exist
127.23.200.212:59325 - accepting connections
I20260807 20:14:33.673843 24355 mini_ranger.cc:161] Starting Ranger...
I20260807 20:14:33.689073 24355 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 20:14:34.185665 24355 mini_ranger.cc:87] Created ranger Postgres database
I20260807 20:14:34.185736 24355 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-admin
2026-08-07 20:14:34,467 [I] DB FLAVOR :POSTGRES
2026-08-07 20:14:34,467 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 20:14:34,467 [I] Checking connection..
2026-08-07 20:14:34,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 20:14:34,735 [I] Checking connection passed.
2026-08-07 20:14:34,735 [I] --------- Verifying version history table ---------
2026-08-07 20:14:34,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 20:14:35,008 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 20:14:35,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 20:14:35,270 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 20:14:35,270 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 20:14:35,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 20:14:35.514 UTC [11328] WARNING: there is no transaction in progress
2026-08-07 20:14:35,521 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 20:14:35,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 20:14:35,786 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 20:14:35,786 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 20:14:35,786 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 20:14:36,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 20:14:36,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2j7r.c.gcp-upstream.internal','N') ;"
2026-08-07 20:14:36,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 20:14:36,780 [I] Table x_portal_user does not exist in database ranger
2026-08-07 20:14:36,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 20:14:37,032 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 20:14:37,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 20:14:37,296 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 20:14:37,531 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 20:14:37,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 20:14:38.518 UTC [11502] WARNING: there is no transaction in progress
2026-08-07 20:14:38.534 UTC [11502] WARNING: there is no transaction in progress
2026-08-07 20:14:38.552 UTC [11502] WARNING: there is no transaction in progress
2026-08-07 20:14:38.568 UTC [11502] WARNING: there is no transaction in progress
2026-08-07 20:14:38.583 UTC [11502] WARNING: there is no transaction in progress
2026-08-07 20:14:38.601 UTC [11502] WARNING: there is no transaction in progress
2026-08-07 20:14:38.654 UTC [11502] WARNING: there is no transaction in progress
2026-08-07 20:14:38.661 UTC [11502] WARNING: there is no transaction in progress
2026-08-07 20:14:38.675 UTC [11502] WARNING: there is no transaction in progress
2026-08-07 20:14:38.686 UTC [11502] WARNING: there is no transaction in progress
2026-08-07 20:14:38.697 UTC [11502] WARNING: there is no transaction in progress
2026-08-07 20:14:38.711 UTC [11502] WARNING: there is no transaction in progress
2026-08-07 20:14:38.721 UTC [11502] WARNING: there is no transaction in progress
2026-08-07 20:14:38.728 UTC [11502] WARNING: there is no transaction in progress
2026-08-07 20:14:38.735 UTC [11502] WARNING: there is no transaction in progress
2026-08-07 20:14:38.742 UTC [11502] WARNING: there is no transaction in progress
2026-08-07 20:14:39,407 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 20:14:39,407 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 20:14:39,667 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 20:14:39,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 20:14:39,925 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 20:14:39,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 20:14:40,192 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 20:14:40,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 20:14:40,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 20:14:40,680 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2j7r.c.gcp-upstream.internal';"
2026-08-07 20:14:40,930 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 20:14:40,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 20:14:41,197 [I] DB_PATCHES have already been applied
I20260807 20:14:41.202718 24355 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-admin:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskISIRft/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 20:14:41.202790 24355 mini_ranger.cc:193] Using host: 127.23.200.212
I20260807 20:14:41.206905 24355 mini_ranger.cc:227] Ranger admin URL: http://127.23.200.212:52885
Aug 07, 2026 8:14:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:14:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:14:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 8:14:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 8:14:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 8:14:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 8:14:42 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 20:14:42.452713 24355 mini_ranger.cc:160] Time spent starting Ranger: real 8.779s user 0.002s sys 0.009s
Aug 07, 2026 8:15:02 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 8:15:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 8:15:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 8:15:02 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 8:15:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 8:15:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 8:15:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 8:15:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 8:15:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 8:15:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 8:15:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 8:15:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 8:15:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 8:15:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 8:15:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 8:15:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 8:15:02 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 8:15:03 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 20:15:04.102746 24355 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.23.200.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.23.200.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.23.200.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/rangerkms_127.23.200.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.23.200.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/rangerkms_127.23.200.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.200.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.23.200.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.200.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/HTTP_127.23.200.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.200.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/HTTP_127.23.200.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 20:15:04 dist-test-slave-2j7r krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133704, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 20:15:04.188627 24355 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 20:15:04.198437 24355 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 20:15:05.059868 24355 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 20:15:05.059952 24355 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-kms
I20260807 20:15:05.059967 24355 mini_ranger_kms.cc:227] Using postgres at 127.23.200.212:59325
2026-08-07 20:15:05,215 [I] DB FLAVOR :POSTGRES
2026-08-07 20:15:05,215 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 20:15:05,215 [I] Checking connection
2026-08-07 20:15:05,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 20:15:05,463 [I] connection success
2026-08-07 20:15:05,463 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 20:15:05,463 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 20:15:05,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 20:15:05,703 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 20:15:05,703 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 20:15:05,704 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 20:15:05,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.200.212:59325/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 20:15:05,962 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 20:15:06.438246 24355 mini_ranger_kms.cc:310] Created kms service
I20260807 20:15:06.631903 24355 mini_ranger_kms.cc:326] Created kudu user
I20260807 20:15:06.674872 24355 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 20:15:06 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133704, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
I20260807 20:15:07.078351 24355 mini_ranger_kms.cc:384] Added ranger policy
I20260807 20:15:07.078562 24355 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-kms:/tmp/dist-test-taskISIRft/build/release/bin/postgresql.jar:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskISIRft/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 20:15:07.078584 24355 mini_ranger_kms.cc:240] Using host: 127.23.200.212
I20260807 20:15:07.082651 24355 mini_ranger_kms.cc:276] Ranger KMS PID: 11792
I20260807 20:15:07.082731 24355 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.23.200.212:56571
Aug 07, 2026 8:15:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:15:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:15:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 8:15:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 8:15:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 8:15:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 20:15:09 dist-test-slave-2j7r krb5kdc[11177](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786133709, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.200.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 20:15:09 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133709, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.200.212@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
I20260807 20:15:09.630153 24355 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.442s user 0.004s sys 0.135s
I20260807 20:15:09.630244 24355 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 20:15:09.630288 24355 mini_ranger_kms.cc:401] 127.23.200.212:56571/kms/v1/keys
Aug 07 20:15:09 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133709, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.200.212@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
Aug 07 20:15:12 dist-test-slave-2j7r krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:15:12.261164 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:37779
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:37779
--ranger_config_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.200.212:56571/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:12.345415 11876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:12.345644 11876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:12.345676 11876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:12.346899 11876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:15:12.346946 11876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:12.346964 11876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:12.346982 11876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:15:12.346997 11876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:15:12.348644 11876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/wal
--ranger_kms_url=127.23.200.212:56571/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:37779
--ranger_config_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:37779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:12.348905 11876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:12.349141 11876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:12.356802 11886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:12.356810 11884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:12.357007 11876 server_base.cc:1034] running on GCE node
W20260807 20:15:12.357146 11883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:12.357501 11876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:12.357800 11876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:12.358960 11876 hybrid_clock.cc:648] HybridClock initialized: now 1786133712358941 us; error 46 us; skew 500 ppm
Aug 07 20:15:12 dist-test-slave-2j7r krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133712, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:12.361506 11876 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:15:12.361955 11876 webserver.cc:460] Webserver started at http://127.23.200.254:37563/ using document root <none> and password file <none>
I20260807 20:15:12.362160 11876 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:12.362201 11876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:12.362325 11876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 20:15:12 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133712, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
I20260807 20:15:12.423862 11876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "95fdd1c82f0a4b6493c26da3b5c0db89"
format_stamp: "Formatted at 2026-08-07 20:15:12 on dist-test-slave-2j7r"
server_key: "dc2cf8654f3ef092f137e03c92fabd74"
server_key_iv: "6762d14a52be9d886778a4242c09ea87"
server_key_version: "kuduclusterkey@0"
I20260807 20:15:12.424337 11876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "95fdd1c82f0a4b6493c26da3b5c0db89"
format_stamp: "Formatted at 2026-08-07 20:15:12 on dist-test-slave-2j7r"
server_key: "dc2cf8654f3ef092f137e03c92fabd74"
server_key_iv: "6762d14a52be9d886778a4242c09ea87"
server_key_version: "kuduclusterkey@0"
I20260807 20:15:12.426373 11876 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 20:15:12.441741 11896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:12.441964 11876 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:15:12.442080 11876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "95fdd1c82f0a4b6493c26da3b5c0db89"
format_stamp: "Formatted at 2026-08-07 20:15:12 on dist-test-slave-2j7r"
server_key: "dc2cf8654f3ef092f137e03c92fabd74"
server_key_iv: "6762d14a52be9d886778a4242c09ea87"
server_key_version: "kuduclusterkey@0"
I20260807 20:15:12.442179 11876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:12.451196 11876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:12.453047 11876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:12.453215 11876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:12.457708 11876 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:37779
I20260807 20:15:12.457737 11997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:37779 every 8 connection(s)
I20260807 20:15:12.458106 11876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:15:12.458477 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 11876
I20260807 20:15:12.458581 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:15:12.458761 24355 external_mini_cluster.cc:1428] Setting key f606d24f6514dab8db1dca16b8d0975e
I20260807 20:15:12.458846 11999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:12.461985 11999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89: Bootstrap starting.
I20260807 20:15:12.462611 11999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:12.462972 11999 log.cc:824] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:12.463765 11999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89: No bootstrap required, opened a new log
I20260807 20:15:12.466200 11999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fdd1c82f0a4b6493c26da3b5c0db89" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37779 } }
I20260807 20:15:12.466383 11999 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:12.466419 11999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95fdd1c82f0a4b6493c26da3b5c0db89, State: Initialized, Role: FOLLOWER
I20260807 20:15:12.466588 11999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fdd1c82f0a4b6493c26da3b5c0db89" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37779 } }
I20260807 20:15:12.466670 11999 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:15:12.466699 11999 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 07 20:15:12 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:12.466807 11999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:12.467450 11999 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fdd1c82f0a4b6493c26da3b5c0db89" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37779 } }
I20260807 20:15:12.467555 11999 leader_election.cc:302] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95fdd1c82f0a4b6493c26da3b5c0db89; no voters:
I20260807 20:15:12.467716 11999 leader_election.cc:288] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:15:12.467759 12004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:15:12.467962 12004 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 1 LEADER]: Becoming Leader. State: Replica: 95fdd1c82f0a4b6493c26da3b5c0db89, State: Running, Role: LEADER
I20260807 20:15:12.468075 11999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:15:12.468089 12004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fdd1c82f0a4b6493c26da3b5c0db89" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37779 } }
I20260807 20:15:12.468544 12004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95fdd1c82f0a4b6493c26da3b5c0db89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fdd1c82f0a4b6493c26da3b5c0db89" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37779 } } }
I20260807 20:15:12.468708 12004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:12.468900 12004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95fdd1c82f0a4b6493c26da3b5c0db89. Latest consensus state: current_term: 1 leader_uuid: "95fdd1c82f0a4b6493c26da3b5c0db89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fdd1c82f0a4b6493c26da3b5c0db89" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37779 } } }
I20260807 20:15:12.469022 12004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:12.469511 11999 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 20:15:12.472425 12016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:15:12.473528 12016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:15:12.474031 12002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:12.459448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56388 (local address 127.23.200.254:37779)
0807 20:15:12.459636 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:12.459647 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:12.459684 (+ 37us) server_negotiation.cc:407] Connection header received
0807 20:15:12.460333 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:12.460343 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:12.460418 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:12.460563 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:12.461365 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:12.462729 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:12.463672 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:12.464283 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:12.464400 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:12.467256 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:12.467261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:12.467261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:12.467274 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:12.469291 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:12.469703 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:12.469707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:12.469707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:12.469804 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:12.469988 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:12.469989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:12.469989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:12.472113 (+ 2124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:12.472939 (+ 826us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:12.473056 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 20:15:12.473667 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260807 20:15:12.477634 12016 catalog_manager.cc:1347] Generated new cluster ID: 5964c29683db43ba815b4ed3aa359d3d
I20260807 20:15:12.477689 12016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:15:12.500046 12016 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:15:12.500871 12016 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:15:12.505352 12016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89: Generated new TSK 0
I20260807 20:15:12.505633 12016 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:15:13.199719 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.200.212:56571/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37779
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:13.301942 12063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:13.302211 12063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:13.302245 12063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:13.303962 12063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:13.304023 12063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:13.304059 12063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:15:13.306267 12063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.23.200.212:56571/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:13.306631 12063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:13.306895 12063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:13.313437 12077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:13.313478 12074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:13.313568 12069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:13.313798 12063 server_base.cc:1034] running on GCE node
I20260807 20:15:13.314009 12063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:13.314239 12063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:13.315373 12063 hybrid_clock.cc:648] HybridClock initialized: now 1786133713315354 us; error 41 us; skew 500 ppm
Aug 07 20:15:13 dist-test-slave-2j7r krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133713, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:13.318063 12063 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:15:13.318481 12063 webserver.cc:460] Webserver started at http://127.23.200.193:33445/ using document root <none> and password file <none>
I20260807 20:15:13.318696 12063 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:13.318743 12063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:13.318869 12063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 20:15:13 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133713, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
I20260807 20:15:13.333827 12063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "5be4493570b04012b6eb36882b77f4f3"
format_stamp: "Formatted at 2026-08-07 20:15:13 on dist-test-slave-2j7r"
server_key: "2afbbf135b146c96fd45b47e3cab5b38"
server_key_iv: "bee1b155095609581e15c5d050ef23d3"
server_key_version: "kuduclusterkey@0"
I20260807 20:15:13.334270 12063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "5be4493570b04012b6eb36882b77f4f3"
format_stamp: "Formatted at 2026-08-07 20:15:13 on dist-test-slave-2j7r"
server_key: "2afbbf135b146c96fd45b47e3cab5b38"
server_key_iv: "bee1b155095609581e15c5d050ef23d3"
server_key_version: "kuduclusterkey@0"
I20260807 20:15:13.336052 12063 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 20:15:13.351709 12087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:13.351888 12063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 20:15:13.351987 12063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "5be4493570b04012b6eb36882b77f4f3"
format_stamp: "Formatted at 2026-08-07 20:15:13 on dist-test-slave-2j7r"
server_key: "2afbbf135b146c96fd45b47e3cab5b38"
server_key_iv: "bee1b155095609581e15c5d050ef23d3"
server_key_version: "kuduclusterkey@0"
I20260807 20:15:13.352079 12063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:13.360415 12063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:13.362615 12063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:13.362814 12063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:13.363090 12063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:13.363507 12063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:15:13.363600 12063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:13.363694 12063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:13.363767 12063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:13.370517 12063 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:36355
I20260807 20:15:13.371126 12063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:15:13.376420 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 12063
I20260807 20:15:13.376516 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:15:13.376721 24355 external_mini_cluster.cc:1428] Setting key 00d19539713e46bcd76f9e5416817112
I20260807 20:15:13.381655 12275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:36355 every 8 connection(s)
Aug 07 20:15:13 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133713, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:13.393272 12283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:13.382442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:60689 (local address 127.23.200.254:37779)
0807 20:15:13.382600 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:13.382602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:13.382822 (+ 220us) server_negotiation.cc:407] Connection header received
0807 20:15:13.383538 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:13.383540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:13.383554 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:13.383604 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:13.384476 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:13.385260 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:13.386775 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:13.387093 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:13.387142 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:13.389736 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:13.389738 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:13.389738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:13.389746 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:13.391544 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:13.392042 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:13.392043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:13.392043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:13.392112 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:13.392389 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:13.392390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:13.392391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:13.392588 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:13.392637 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:13.393024 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 20:15:13.393163 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260807 20:15:13.393777 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:13.382400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37779 (local address 127.23.200.193:60689)
0807 20:15:13.382677 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:13.382732 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:13.383404 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:13.383804 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:13.383809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:13.384087 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:13.384367 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:13.384374 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:13.385379 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:13.385380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:13.386667 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:13.386670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:13.387257 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:13.387258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:13.387407 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:13.387966 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:13.387975 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:13.389615 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:13.391636 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:13.391643 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:13.391645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:13.391949 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:13.392180 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:13.392181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:13.392181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:13.392308 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:13.392685 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:13.392692 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:13.392855 (+ 163us) client_negotiation.cc:769] Sending connection context
0807 20:15:13.393380 (+ 525us) client_negotiation.cc:240] Negotiation successful
0807 20:15:13.393547 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":105,"threads_started":1}
I20260807 20:15:13.394388 12276 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37779
I20260807 20:15:13.394501 12276 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:13.394716 12276 heartbeater.cc:511] Master 127.23.200.254:37779 requested a full tablet report, sending...
I20260807 20:15:13.395352 11924 ts_manager.cc:194] Registered new tserver with Master: 5be4493570b04012b6eb36882b77f4f3 (127.23.200.193:36355)
I20260807 20:15:13.396270 11924 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:60689
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:15:13.420189 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.200.212:56571/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37779
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:13.523685 12285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:13.523896 12285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:13.523931 12285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:13.525120 12285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:13.525166 12285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:13.525200 12285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:15:13.526723 12285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.23.200.212:56571/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:13.526984 12285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:13.527225 12285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:13.532622 12300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:13.532618 12296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:13.532620 12295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:13.533973 12285 server_base.cc:1034] running on GCE node
I20260807 20:15:13.534183 12285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:13.534446 12285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:13.535589 12285 hybrid_clock.cc:648] HybridClock initialized: now 1786133713535572 us; error 34 us; skew 500 ppm
Aug 07 20:15:13 dist-test-slave-2j7r krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133713, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:13.538686 12285 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:15:13.539150 12285 webserver.cc:460] Webserver started at http://127.23.200.194:44503/ using document root <none> and password file <none>
I20260807 20:15:13.539400 12285 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:13.539453 12285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:13.539585 12285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 20:15:13 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133713, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
Aug 07 20:15:13 dist-test-slave-2j7r krb5kdc[11177](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786133713, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:13.554237 12285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "e8c411e4002e4d1fbddeed25b307612f"
format_stamp: "Formatted at 2026-08-07 20:15:13 on dist-test-slave-2j7r"
server_key: "04749dae98e61ade0cfbe2d67aa03290"
server_key_iv: "de6d63b8c54a7253364ac0fdbb42deac"
server_key_version: "kuduclusterkey@0"
I20260807 20:15:13.554697 12285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "e8c411e4002e4d1fbddeed25b307612f"
format_stamp: "Formatted at 2026-08-07 20:15:13 on dist-test-slave-2j7r"
server_key: "04749dae98e61ade0cfbe2d67aa03290"
server_key_iv: "de6d63b8c54a7253364ac0fdbb42deac"
server_key_version: "kuduclusterkey@0"
I20260807 20:15:13.562141 12285 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.003s
I20260807 20:15:13.581041 12315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:13.584964 12285 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260807 20:15:13.585129 12285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "e8c411e4002e4d1fbddeed25b307612f"
format_stamp: "Formatted at 2026-08-07 20:15:13 on dist-test-slave-2j7r"
server_key: "04749dae98e61ade0cfbe2d67aa03290"
server_key_iv: "de6d63b8c54a7253364ac0fdbb42deac"
server_key_version: "kuduclusterkey@0"
I20260807 20:15:13.585246 12285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:13.619369 12285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:13.621632 12285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:13.621843 12285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:13.622141 12285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:13.622601 12285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:15:13.622658 12285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:13.622692 12285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:13.622746 12285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:13.630265 12285 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:44405
I20260807 20:15:13.630792 12285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:15:13.631487 12505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:44405 every 8 connection(s)
I20260807 20:15:13.639204 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 12285
I20260807 20:15:13.639310 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:15:13.639506 24355 external_mini_cluster.cc:1428] Setting key 2e5eb784b2cc30f426d1c8fc508a18ba
Aug 07 20:15:13 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133713, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:13.644232 12283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:13.632494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:59483 (local address 127.23.200.254:37779)
0807 20:15:13.632542 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:13.632544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:13.633376 (+ 832us) server_negotiation.cc:407] Connection header received
0807 20:15:13.634113 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:13.634114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:13.634132 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:13.634221 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:13.634962 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:13.635778 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:13.637341 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:13.637632 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:13.637678 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:13.640695 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:13.640697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:13.640697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:13.640706 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:13.642469 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:13.643040 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:13.643042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:13.643042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:13.643108 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:13.643413 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:13.643415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:13.643415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:13.643562 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:13.643607 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:13.644022 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 20:15:13.644124 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:15:13.645079 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:13.632200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37779 (local address 127.23.200.194:59483)
0807 20:15:13.633227 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:13.633279 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:13.633984 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:13.634298 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:13.634303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:13.634544 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:13.634841 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:13.634847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:13.635914 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:13.635916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:13.637238 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:13.637241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:13.637782 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:13.637782 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:13.637939 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:13.638519 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:13.638530 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:13.640587 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:13.642569 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:13.642577 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:13.642580 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:13.642892 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:13.643181 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:13.643182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:13.643182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:13.643329 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:13.643655 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:13.643662 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:13.643838 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 20:15:13.644603 (+ 765us) client_negotiation.cc:240] Negotiation successful
0807 20:15:13.644782 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":976,"thread_start_us":88,"threads_started":1}
I20260807 20:15:13.645835 12507 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37779
I20260807 20:15:13.645974 12507 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:13.646188 12507 heartbeater.cc:511] Master 127.23.200.254:37779 requested a full tablet report, sending...
I20260807 20:15:13.646610 11924 ts_manager.cc:194] Registered new tserver with Master: e8c411e4002e4d1fbddeed25b307612f (127.23.200.194:44405)
I20260807 20:15:13.647100 11924 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:59483
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:15:13.680588 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.200.212:56571/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37779
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:13.811962 12516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:13.812323 12516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:13.812402 12516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:13.814307 12516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:13.814419 12516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:13.814500 12516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:15:13.816792 12516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.23.200.212:56571/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:37779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:13.817303 12516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:13.817646 12516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:13.824648 12528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:13.824648 12529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:13.825075 12536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:13.826120 12516 server_base.cc:1034] running on GCE node
I20260807 20:15:13.826450 12516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:13.826725 12516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:13.827881 12516 hybrid_clock.cc:648] HybridClock initialized: now 1786133713827860 us; error 34 us; skew 500 ppm
Aug 07 20:15:13 dist-test-slave-2j7r krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133713, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:13.831210 12516 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:15:13.831723 12516 webserver.cc:460] Webserver started at http://127.23.200.195:39793/ using document root <none> and password file <none>
I20260807 20:15:13.832008 12516 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:13.832060 12516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:13.832187 12516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 20:15:13 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133713, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
I20260807 20:15:13.854676 12516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "b2151dfe505d41609a85c8d16aa3a6bc"
format_stamp: "Formatted at 2026-08-07 20:15:13 on dist-test-slave-2j7r"
server_key: "1e54d59d9a24e651b445cb9809ad897f"
server_key_iv: "9ae533acaff17ff95cb61b332d7a8d8d"
server_key_version: "kuduclusterkey@0"
I20260807 20:15:13.857010 12516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "b2151dfe505d41609a85c8d16aa3a6bc"
format_stamp: "Formatted at 2026-08-07 20:15:13 on dist-test-slave-2j7r"
server_key: "1e54d59d9a24e651b445cb9809ad897f"
server_key_iv: "9ae533acaff17ff95cb61b332d7a8d8d"
server_key_version: "kuduclusterkey@0"
I20260807 20:15:13.862320 12516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260807 20:15:13.881213 12545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:13.885238 12516 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260807 20:15:13.885385 12516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "b2151dfe505d41609a85c8d16aa3a6bc"
format_stamp: "Formatted at 2026-08-07 20:15:13 on dist-test-slave-2j7r"
server_key: "1e54d59d9a24e651b445cb9809ad897f"
server_key_iv: "9ae533acaff17ff95cb61b332d7a8d8d"
server_key_version: "kuduclusterkey@0"
I20260807 20:15:13.885489 12516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:13.910164 12516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:13.915311 12516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:13.915458 12516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:13.915712 12516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:13.916090 12516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:15:13.916128 12516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:13.916158 12516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:13.916208 12516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:13.922421 12516 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:33639
I20260807 20:15:13.922931 12516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:15:13.923168 12730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:33639 every 8 connection(s)
I20260807 20:15:13.924942 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 12516
I20260807 20:15:13.925051 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:15:13.925273 24355 external_mini_cluster.cc:1428] Setting key 347effb7b00ecc7b9e6fe1b22387a355
Aug 07 20:15:13 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133713, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:13.933969 12283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:13.925843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:54673 (local address 127.23.200.254:37779)
0807 20:15:13.925890 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:13.925893 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:13.926157 (+ 264us) server_negotiation.cc:407] Connection header received
0807 20:15:13.926837 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:13.926838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:13.926849 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:13.926878 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:13.927546 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:13.928055 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:13.929067 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:13.929319 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:13.929376 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:13.931365 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:13.931367 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:13.931368 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:13.931376 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:13.932629 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:13.933006 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:13.933007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:13.933007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:13.933060 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:13.933336 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:13.933336 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:13.933337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:13.933451 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:13.933509 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:13.933786 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 20:15:13.933874 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:15:13.934136 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:13.925832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37779 (local address 127.23.200.195:54673)
0807 20:15:13.926024 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:13.926067 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:13.926712 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:13.927061 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:13.927065 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:13.927256 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:13.927455 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:13.927460 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:13.928145 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:13.928146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:13.928985 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:13.928988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:13.929410 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:13.929411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:13.929536 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:13.930023 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:13.930031 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:13.931270 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:13.932706 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:13.932714 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:13.932716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:13.932940 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:13.933153 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:13.933154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:13.933154 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:13.933271 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:13.933507 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:13.933515 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:13.933656 (+ 141us) client_negotiation.cc:769] Sending connection context
0807 20:15:13.933750 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 20:15:13.933899 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":91,"threads_started":1}
I20260807 20:15:13.934576 12731 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37779
I20260807 20:15:13.934677 12731 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:13.934865 12731 heartbeater.cc:511] Master 127.23.200.254:37779 requested a full tablet report, sending...
I20260807 20:15:13.935248 11924 ts_manager.cc:194] Registered new tserver with Master: b2151dfe505d41609a85c8d16aa3a6bc (127.23.200.195:33639)
I20260807 20:15:13.935614 11924 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:54673
I20260807 20:15:13.937573 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:15:13.938129 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 12063
I20260807 20:15:13.985344 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 12285
I20260807 20:15:13.995060 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 12516
I20260807 20:15:14.005179 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 11876
I20260807 20:15:14.024063 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:37779
--webserver_interface=127.23.200.254
--webserver_port=37563
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:37779
--ranger_config_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.200.212:56571/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.200.212:56571/kms with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:14.106617 12738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:14.106824 12738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:14.106854 12738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:14.107960 12738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:15:14.108001 12738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:14.108021 12738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:14.108035 12738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:15:14.108052 12738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:15:14.109709 12738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.200.212:56571/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:37779
--ranger_config_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:37779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=37563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:14.109964 12738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:14.110162 12738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:14.116773 12747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:14.116860 12738 server_base.cc:1034] running on GCE node
W20260807 20:15:14.116964 12749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:14.116824 12746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:14.117301 12738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:14.117632 12738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:14.118793 12738 hybrid_clock.cc:648] HybridClock initialized: now 1786133714118772 us; error 37 us; skew 500 ppm
Aug 07 20:15:14 dist-test-slave-2j7r krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133714, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:14.125852 12738 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:15:14.126385 12738 webserver.cc:460] Webserver started at http://127.23.200.254:37563/ using document root <none> and password file <none>
I20260807 20:15:14.126606 12738 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:14.126659 12738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 20:15:14 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133714, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
I20260807 20:15:14.169760 12738 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:15:14.170728 12760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:14.170907 12738 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:15:14.170996 12738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "95fdd1c82f0a4b6493c26da3b5c0db89"
format_stamp: "Formatted at 2026-08-07 20:15:12 on dist-test-slave-2j7r"
server_key: "dc2cf8654f3ef092f137e03c92fabd74"
server_key_iv: "6762d14a52be9d886778a4242c09ea87"
server_key_version: "kuduclusterkey@0"
I20260807 20:15:14.171353 12738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:14.189437 12738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:14.191406 12738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:14.191565 12738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:14.196386 12738 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:37779
I20260807 20:15:14.196455 12848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:37779 every 8 connection(s)
I20260807 20:15:14.196834 12738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:15:14.198118 12850 sys_catalog.cc:263] Verifying existing consensus state
I20260807 20:15:14.198783 12850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89: Bootstrap starting.
I20260807 20:15:14.200399 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 12738
I20260807 20:15:14.200722 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.200.212:56571/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:36355
--local_ip_for_outbound_sockets=127.23.200.193
--tserver_master_addrs=127.23.200.254:37779
--webserver_port=33445
--webserver_interface=127.23.200.193
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.200.212:56571/kms with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:15:14.207965 12850 log.cc:824] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:14.210364 12850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:15:14.210680 12850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89: Bootstrap complete.
I20260807 20:15:14.214162 12850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fdd1c82f0a4b6493c26da3b5c0db89" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37779 } }
I20260807 20:15:14.214490 12850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95fdd1c82f0a4b6493c26da3b5c0db89, State: Initialized, Role: FOLLOWER
I20260807 20:15:14.214663 12850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fdd1c82f0a4b6493c26da3b5c0db89" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37779 } }
I20260807 20:15:14.214735 12850 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:15:14.214780 12850 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:15:14.214825 12850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 1 FOLLOWER]: Advancing to term 2
I20260807 20:15:14.216154 12850 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fdd1c82f0a4b6493c26da3b5c0db89" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37779 } }
I20260807 20:15:14.216271 12850 leader_election.cc:302] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95fdd1c82f0a4b6493c26da3b5c0db89; no voters:
I20260807 20:15:14.216444 12850 leader_election.cc:288] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 20:15:14.216523 12854 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 20:15:14.216697 12850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:15:14.216717 12854 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [term 2 LEADER]: Becoming Leader. State: Replica: 95fdd1c82f0a4b6493c26da3b5c0db89, State: Running, Role: LEADER
I20260807 20:15:14.216945 12854 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fdd1c82f0a4b6493c26da3b5c0db89" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37779 } }
I20260807 20:15:14.217276 12855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "95fdd1c82f0a4b6493c26da3b5c0db89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fdd1c82f0a4b6493c26da3b5c0db89" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37779 } } }
I20260807 20:15:14.217351 12855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:14.217661 12859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95fdd1c82f0a4b6493c26da3b5c0db89. Latest consensus state: current_term: 2 leader_uuid: "95fdd1c82f0a4b6493c26da3b5c0db89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95fdd1c82f0a4b6493c26da3b5c0db89" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37779 } } }
I20260807 20:15:14.217732 12870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:15:14.217748 12859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89 [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:14.218210 12850 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 20:15:14.218402 12870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:15:14.218611 12870 catalog_manager.cc:1259] Loaded cluster ID: 5964c29683db43ba815b4ed3aa359d3d
I20260807 20:15:14.218676 12870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:15:14.223667 12870 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:15:14.224025 12870 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89: Loaded TSK: 0
I20260807 20:15:14.224361 12870 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 20:15:14.315670 12852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:14.315863 12852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:14.315896 12852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:14.317159 12852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:14.317200 12852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:14.317245 12852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:15:14.318837 12852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.200.212:56571/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:36355
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=33445
--tserver_master_addrs=127.23.200.254:37779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:14.319064 12852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:14.319301 12852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:14.324326 12909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:14.324340 12905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:14.328445 12906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:14.328482 12852 server_base.cc:1034] running on GCE node
I20260807 20:15:14.328739 12852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:14.329000 12852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:14.330174 12852 hybrid_clock.cc:648] HybridClock initialized: now 1786133714330163 us; error 46 us; skew 500 ppm
Aug 07 20:15:14 dist-test-slave-2j7r krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133714, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:14.333117 12852 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:15:14.333652 12852 webserver.cc:460] Webserver started at http://127.23.200.193:33445/ using document root <none> and password file <none>
I20260807 20:15:14.333912 12852 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:14.333984 12852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 20:15:14 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133714, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
I20260807 20:15:14.356540 12852 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:15:14.358978 12926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:14.359476 12852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 20:15:14.359586 12852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "5be4493570b04012b6eb36882b77f4f3"
format_stamp: "Formatted at 2026-08-07 20:15:13 on dist-test-slave-2j7r"
server_key: "2afbbf135b146c96fd45b47e3cab5b38"
server_key_iv: "bee1b155095609581e15c5d050ef23d3"
server_key_version: "kuduclusterkey@0"
I20260807 20:15:14.360091 12852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:14.372080 12852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:14.374223 12852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:14.374374 12852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:14.374631 12852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:14.374996 12852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:15:14.375039 12852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:14.375077 12852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:14.375100 12852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:14.382457 12852 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:36355
I20260807 20:15:14.382532 13107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:36355 every 8 connection(s)
I20260807 20:15:14.382964 12852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:15:14.389056 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 12852
I20260807 20:15:14.389439 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.200.212:56571/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:44405
--local_ip_for_outbound_sockets=127.23.200.194
--tserver_master_addrs=127.23.200.254:37779
--webserver_port=44503
--webserver_interface=127.23.200.194
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.200.212:56571/kms with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 20:15:14 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133714, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:14.399703 13112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:14.383796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37779 (local address 127.23.200.193:43805)
0807 20:15:14.384012 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:14.384062 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:14.384945 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:14.385672 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:14.385677 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:14.385927 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:14.386232 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:14.386239 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:14.387669 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:14.387671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:14.391431 (+ 3760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:14.391436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:14.392368 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:14.392370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:14.392524 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:14.393196 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:14.393207 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:14.395337 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:14.397631 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:14.397640 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:14.397643 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:14.397954 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:14.398272 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:14.398273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:14.398274 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:14.398432 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:14.398961 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:14.398968 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:14.399137 (+ 169us) client_negotiation.cc:769] Sending connection context
0807 20:15:14.399254 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 20:15:14.399438 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":105,"threads_started":1}
I20260807 20:15:14.400424 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:14.384226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:43805 (local address 127.23.200.254:37779)
0807 20:15:14.385198 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:14.385208 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:14.385246 (+ 38us) server_negotiation.cc:407] Connection header received
0807 20:15:14.385393 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:14.385399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:14.385455 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:14.385598 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:14.386362 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:14.387552 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:14.391548 (+ 3996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:14.392232 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:14.392362 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:14.395490 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:14.395496 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:14.395496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:14.395510 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:14.397538 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:14.398064 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:14.398069 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:14.398069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:14.398194 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:14.398508 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:14.398509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:14.398510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:14.398845 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:14.398928 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:14.400064 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0807 20:15:14.400201 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":911,"thread_start_us":82,"threads_started":1}
I20260807 20:15:14.401206 13108 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37779
I20260807 20:15:14.401317 13108 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:14.401538 13108 heartbeater.cc:511] Master 127.23.200.254:37779 requested a full tablet report, sending...
W20260807 20:15:14.402536 13108 heartbeater.cc:644] Failed to heartbeat to 127.23.200.254:37779 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 20:15:14.402611 13108 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:14.402681 13108 heartbeater.cc:511] Master 127.23.200.254:37779 requested a full tablet report, sending...
I20260807 20:15:14.403025 13108 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:14.403091 13108 heartbeater.cc:511] Master 127.23.200.254:37779 requested a full tablet report, sending...
W20260807 20:15:14.403416 13108 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 20:15:14.499034 13114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:14.499325 13114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:14.499363 13114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:14.500784 13114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:14.500885 13114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:14.500957 13114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:15:14.502887 13114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.200.212:56571/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:44405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=44503
--tserver_master_addrs=127.23.200.254:37779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:14.503139 13114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:14.503381 13114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:14.508949 13120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:14.509327 13127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:14.509859 13114 server_base.cc:1034] running on GCE node
W20260807 20:15:14.513235 13121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:14.513548 13114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:14.513860 13114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:14.515038 13114 hybrid_clock.cc:648] HybridClock initialized: now 1786133714515007 us; error 44 us; skew 500 ppm
Aug 07 20:15:14 dist-test-slave-2j7r krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133714, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:14.518560 13114 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:15:14.519177 13114 webserver.cc:460] Webserver started at http://127.23.200.194:44503/ using document root <none> and password file <none>
I20260807 20:15:14.519471 13114 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:14.519572 13114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 20:15:14 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133714, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
I20260807 20:15:14.544806 13114 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:15:14.546738 13146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:14.546985 13114 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260807 20:15:14.547086 13114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "e8c411e4002e4d1fbddeed25b307612f"
format_stamp: "Formatted at 2026-08-07 20:15:13 on dist-test-slave-2j7r"
server_key: "04749dae98e61ade0cfbe2d67aa03290"
server_key_iv: "de6d63b8c54a7253364ac0fdbb42deac"
server_key_version: "kuduclusterkey@0"
I20260807 20:15:14.547477 13114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:14.557561 13114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:14.559899 13114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:14.560050 13114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:14.560318 13114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:14.560695 13114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:15:14.560747 13114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:14.560778 13114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:14.560807 13114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:14.568178 13114 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:44405
I20260807 20:15:14.568673 13114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:15:14.569336 13317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:44405 every 8 connection(s)
Aug 07 20:15:14 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133714, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:14.576869 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 13114
I20260807 20:15:14.577163 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.200.212:56571/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:33639
--local_ip_for_outbound_sockets=127.23.200.195
--tserver_master_addrs=127.23.200.254:37779
--webserver_port=39793
--webserver_interface=127.23.200.195
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.200.212:56571/kms with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:15:14.578822 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:14.569538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:57299 (local address 127.23.200.254:37779)
0807 20:15:14.569581 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:14.569583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:14.569841 (+ 258us) server_negotiation.cc:407] Connection header received
0807 20:15:14.570311 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:14.570312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:14.570330 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:14.570401 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:14.571041 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:14.571816 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:14.572871 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:14.573268 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:14.573350 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:14.575368 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:14.575369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:14.575369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:14.575377 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:14.577144 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:14.577633 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:14.577634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:14.577635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:14.577700 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:14.578028 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:14.578030 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:14.578030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:14.578171 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:14.578220 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:14.578624 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 20:15:14.578716 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:15:14.579360 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:14.569576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37779 (local address 127.23.200.194:57299)
0807 20:15:14.569725 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:14.569762 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:14.570198 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:14.570539 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:14.570545 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:14.570726 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:14.570947 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:14.570953 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:14.571929 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:14.571931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:14.572774 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:14.572776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:14.573379 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:14.573381 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:14.573501 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:14.574012 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:14.574020 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:14.575269 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:14.577234 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:14.577241 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:14.577244 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:14.577539 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:14.577797 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:14.577798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:14.577798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:14.577940 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:14.578275 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:14.578283 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:14.578449 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 20:15:14.578958 (+ 509us) client_negotiation.cc:240] Negotiation successful
0807 20:15:14.579125 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260807 20:15:14.579864 13320 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37779
I20260807 20:15:14.579972 13320 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:14.580227 13320 heartbeater.cc:511] Master 127.23.200.254:37779 requested a full tablet report, sending...
W20260807 20:15:14.581071 13320 heartbeater.cc:644] Failed to heartbeat to 127.23.200.254:37779 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 20:15:14.581229 13320 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:14.581341 13320 heartbeater.cc:511] Master 127.23.200.254:37779 requested a full tablet report, sending...
I20260807 20:15:14.581786 13320 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:14.581933 13320 heartbeater.cc:511] Master 127.23.200.254:37779 requested a full tablet report, sending...
W20260807 20:15:14.582432 13320 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 20:15:14.680781 13326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:14.681064 13326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:14.681124 13326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:14.682878 13326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:14.682945 13326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:14.682986 13326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:15:14.685266 13326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.200.212:56571/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:33639
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=39793
--tserver_master_addrs=127.23.200.254:37779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:14.685544 13326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:14.685905 13326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:14.692402 13334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:14.692631 13333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:14.692835 13338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:14.693358 13326 server_base.cc:1034] running on GCE node
I20260807 20:15:14.693602 13326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:14.693910 13326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:14.695125 13326 hybrid_clock.cc:648] HybridClock initialized: now 1786133714695098 us; error 41 us; skew 500 ppm
Aug 07 20:15:14 dist-test-slave-2j7r krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133714, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:14.698536 13326 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:15:14.699162 13326 webserver.cc:460] Webserver started at http://127.23.200.195:39793/ using document root <none> and password file <none>
I20260807 20:15:14.699461 13326 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:14.699550 13326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 20:15:14 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133714, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
I20260807 20:15:14.727197 13326 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:15:14.728384 13356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:14.728622 13326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:15:14.728727 13326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "b2151dfe505d41609a85c8d16aa3a6bc"
format_stamp: "Formatted at 2026-08-07 20:15:13 on dist-test-slave-2j7r"
server_key: "1e54d59d9a24e651b445cb9809ad897f"
server_key_iv: "9ae533acaff17ff95cb61b332d7a8d8d"
server_key_version: "kuduclusterkey@0"
I20260807 20:15:14.729202 13326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:14.757740 13326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:14.759698 13326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:14.759826 13326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:14.760100 13326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:14.760488 13326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:15:14.760527 13326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:14.760550 13326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:14.760560 13326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:14.767388 13326 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:33639
I20260807 20:15:14.767469 13529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:33639 every 8 connection(s)
I20260807 20:15:14.767855 13326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
Aug 07 20:15:14 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133714, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:14.776309 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 13326
I20260807 20:15:14.776813 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:14.768598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:42337 (local address 127.23.200.254:37779)
0807 20:15:14.768647 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:14.768650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:14.768874 (+ 224us) server_negotiation.cc:407] Connection header received
0807 20:15:14.769438 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:14.769440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:14.769453 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:14.769496 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:14.770037 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:14.770521 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:14.771583 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:14.771788 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:14.771845 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:14.773867 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:14.773868 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:14.773869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:14.773884 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:14.775459 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:14.775826 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:14.775827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:14.775828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:14.775893 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:14.776116 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:14.776117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:14.776117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:14.776233 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:14.776291 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:14.776627 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 20:15:14.776720 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:15:14.777460 13536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:14.768599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37779 (local address 127.23.200.195:42337)
0807 20:15:14.768750 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:14.768792 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:14.769321 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:14.769580 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:14.769584 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:14.769752 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:14.769944 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:14.769949 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:14.770608 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:14.770609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:14.771495 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:14.771498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:14.771876 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:14.771877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:14.771996 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:14.772497 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:14.772506 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:14.773789 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:14.775534 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:14.775541 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:14.775543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:14.775759 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:14.775948 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:14.775949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:14.775949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:14.776060 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:14.776289 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:14.776296 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:14.776465 (+ 169us) client_negotiation.cc:769] Sending connection context
0807 20:15:14.777010 (+ 545us) client_negotiation.cc:240] Negotiation successful
0807 20:15:14.777205 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260807 20:15:14.778019 13530 heartbeater.cc:347] Connected to a master server at 127.23.200.254:37779
I20260807 20:15:14.778115 13530 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:14.778290 13530 heartbeater.cc:511] Master 127.23.200.254:37779 requested a full tablet report, sending...
W20260807 20:15:14.778910 13530 heartbeater.cc:644] Failed to heartbeat to 127.23.200.254:37779 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 20:15:14.778985 13530 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:14.779043 13530 heartbeater.cc:511] Master 127.23.200.254:37779 requested a full tablet report, sending...
I20260807 20:15:14.779392 13530 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:14.779464 13530 heartbeater.cc:511] Master 127.23.200.254:37779 requested a full tablet report, sending...
W20260807 20:15:14.779759 13530 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 20:15:14.784499 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:14.776972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56394 (local address 127.23.200.254:37779)
0807 20:15:14.777009 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:14.777010 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:14.777262 (+ 252us) server_negotiation.cc:407] Connection header received
0807 20:15:14.777352 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:14.777353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:14.777364 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:14.777392 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:14.778162 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:14.778837 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:14.780213 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:14.780499 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:14.780571 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:14.781973 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:14.781974 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:14.781975 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:14.781982 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:14.783408 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:14.783739 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:14.783740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:14.783740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:14.783816 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:14.783983 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:14.783983 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:14.783984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:14.784114 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:14.784288 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:14.784330 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 20:15:14.784408 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Aug 07 20:15:15 dist-test-slave-2j7r krb5kdc[11177](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786133715, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:15.403607 13108 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:15.403780 13108 heartbeater.cc:511] Master 127.23.200.254:37779 requested a full tablet report, sending...
I20260807 20:15:15.404281 12784 ts_manager.cc:194] Registered new tserver with Master: 5be4493570b04012b6eb36882b77f4f3 (127.23.200.193:36355)
I20260807 20:15:15.405365 12784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:43805
I20260807 20:15:15.582672 13320 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:15.582840 13320 heartbeater.cc:511] Master 127.23.200.254:37779 requested a full tablet report, sending...
I20260807 20:15:15.583159 12784 ts_manager.cc:194] Registered new tserver with Master: e8c411e4002e4d1fbddeed25b307612f (127.23.200.194:44405)
I20260807 20:15:15.583699 12784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:57299
Aug 07 20:15:15 dist-test-slave-2j7r krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133715, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for HTTP/127.23.200.212@KRBTEST.COM
I20260807 20:15:15.779932 13530 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:15.780087 13530 heartbeater.cc:511] Master 127.23.200.254:37779 requested a full tablet report, sending...
I20260807 20:15:15.780400 12784 ts_manager.cc:194] Registered new tserver with Master: b2151dfe505d41609a85c8d16aa3a6bc (127.23.200.195:33639)
I20260807 20:15:15.780793 12784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:42337
I20260807 20:15:15.787165 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:15:15.794741 13574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:15.788114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56410 (local address 127.23.200.254:37779)
0807 20:15:15.788237 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:15.788239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:15.788276 (+ 37us) server_negotiation.cc:407] Connection header received
0807 20:15:15.788355 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:15.788356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:15.788367 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:15.788422 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:15.789155 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.789655 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.790695 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.790902 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.790952 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:15.792314 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:15.792316 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:15.792316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:15.792322 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:15.793707 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:15.794074 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:15.794074 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:15.794075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:15.794123 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:15.794301 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:15.794301 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:15.794301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:15.794396 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:15.794466 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:15.794605 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 20:15:15.794668 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":77,"threads_started":1}
I20260807 20:15:15.796653 12784 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:15:15.798514 12784 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:15:15.809764 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:15.804230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:33639 (local address 127.0.0.1:60080)
0807 20:15:15.804969 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:15.805014 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:15.805079 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:15.805541 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:15.805545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:15.805547 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:15.805630 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:15.805631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.807089 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.807090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:15.808338 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:15.808342 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.809421 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.809423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:15.809518 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:15.809533 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 20:15:15.809603 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 20:15:15.809654 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":90,"threads_started":1}
I20260807 20:15:15.810382 13593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:15.804810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60080 (local address 127.23.200.195:33639)
0807 20:15:15.805287 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:15.805293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:15.805303 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:15:15.805340 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:15.805342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:15.805359 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:15.805428 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:15.805727 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.806818 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.808448 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.809241 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.809325 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:15.809383 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:15.810250 (+ 867us) server_negotiation.cc:299] Negotiation successful
0807 20:15:15.810300 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":46,"threads_started":1}
I20260807 20:15:15.810580 13585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:15.804489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:44405 (local address 127.0.0.1:49282)
0807 20:15:15.805741 (+ 1252us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:15.805751 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:15.805787 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:15.806139 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:15.806141 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:15.806142 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:15.806200 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:15.806202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.807702 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.807703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:15.808859 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:15.808862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.810347 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.810348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:15.810413 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:15.810417 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:15:15.810462 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 20:15:15.810502 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1236,"mutex_wait_us":13,"thread_start_us":82,"threads_started":1}
I20260807 20:15:15.811003 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:15.804608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49282 (local address 127.23.200.194:44405)
0807 20:15:15.805894 (+ 1286us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:15.805899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:15.805909 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:15:15.805944 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:15.805946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:15.805981 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:15.806053 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:15.806273 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.807603 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.809111 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.810182 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.810256 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:15.810310 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:15.810884 (+ 574us) server_negotiation.cc:299] Negotiation successful
0807 20:15:15.810928 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1248,"thread_start_us":82,"threads_started":1}
I20260807 20:15:15.811345 13431 tablet_service.cc:1467] Processing CreateTablet for tablet 92a45f0c55ac44a6ba4341c3f30a2f79 (DEFAULT_TABLE table=test-table [id=1347b072718045409306906685a729e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:15:15.811777 13431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92a45f0c55ac44a6ba4341c3f30a2f79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:15.811823 13217 tablet_service.cc:1467] Processing CreateTablet for tablet 92a45f0c55ac44a6ba4341c3f30a2f79 (DEFAULT_TABLE table=test-table [id=1347b072718045409306906685a729e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:15:15.812186 13217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92a45f0c55ac44a6ba4341c3f30a2f79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:15.815313 13584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:15.804663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:36355 (local address 127.0.0.1:35416)
0807 20:15:15.808983 (+ 4320us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:15.808992 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:15.809031 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:15.811455 (+ 2424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:15.811457 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:15.811459 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:15.811500 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:15.811501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.813143 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.813145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:15.814165 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:15.814169 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.815115 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.815116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:15.815175 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:15.815180 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:15:15.815207 (+ 27us) client_negotiation.cc:240] Negotiation successful
0807 20:15:15.815243 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4303,"thread_start_us":75,"threads_started":1}
I20260807 20:15:15.815899 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:15.804868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35416 (local address 127.23.200.193:36355)
0807 20:15:15.811279 (+ 6411us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:15.811285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:15.811296 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:15:15.811331 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:15.811333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:15.811348 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:15.811573 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:15.811695 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.813001 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.814270 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.815013 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.815416 (+ 403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:15.815757 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:15.815784 (+ 27us) server_negotiation.cc:299] Negotiation successful
0807 20:15:15.815821 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6373,"thread_start_us":69,"threads_started":1}
I20260807 20:15:15.816732 12985 tablet_service.cc:1467] Processing CreateTablet for tablet 92a45f0c55ac44a6ba4341c3f30a2f79 (DEFAULT_TABLE table=test-table [id=1347b072718045409306906685a729e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:15:15.817107 12985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92a45f0c55ac44a6ba4341c3f30a2f79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:15.821066 13599 tablet_bootstrap.cc:492] T 92a45f0c55ac44a6ba4341c3f30a2f79 P 5be4493570b04012b6eb36882b77f4f3: Bootstrap starting.
I20260807 20:15:15.821874 13599 tablet_bootstrap.cc:654] T 92a45f0c55ac44a6ba4341c3f30a2f79 P 5be4493570b04012b6eb36882b77f4f3: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:15.822223 13599 log.cc:824] T 92a45f0c55ac44a6ba4341c3f30a2f79 P 5be4493570b04012b6eb36882b77f4f3: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:15.823071 13597 tablet_bootstrap.cc:492] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc: Bootstrap starting.
I20260807 20:15:15.823626 13597 tablet_bootstrap.cc:654] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:15.823892 13597 log.cc:824] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:15.824247 13598 tablet_bootstrap.cc:492] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f: Bootstrap starting.
I20260807 20:15:15.824633 13597 tablet_bootstrap.cc:492] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc: No bootstrap required, opened a new log
I20260807 20:15:15.824695 13597 ts_tablet_manager.cc:1397] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:15:15.826438 13597 raft_consensus.cc:348] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2151dfe505d41609a85c8d16aa3a6bc" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 33639 } } peers { permanent_uuid: "e8c411e4002e4d1fbddeed25b307612f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44405 } } peers { permanent_uuid: "5be4493570b04012b6eb36882b77f4f3" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 36355 } }
I20260807 20:15:15.826578 13597 raft_consensus.cc:374] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:15.826609 13597 raft_consensus.cc:729] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2151dfe505d41609a85c8d16aa3a6bc, State: Initialized, Role: FOLLOWER
I20260807 20:15:15.826769 13597 consensus_queue.cc:260] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2151dfe505d41609a85c8d16aa3a6bc" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 33639 } } peers { permanent_uuid: "e8c411e4002e4d1fbddeed25b307612f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44405 } } peers { permanent_uuid: "5be4493570b04012b6eb36882b77f4f3" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 36355 } }
I20260807 20:15:15.827026 13597 ts_tablet_manager.cc:1428] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:15:15.827154 13530 heartbeater.cc:503] Master 127.23.200.254:37779 was elected leader, sending a full tablet report...
I20260807 20:15:15.827808 13598 tablet_bootstrap.cc:654] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:15.828121 13598 log.cc:824] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:15.828861 13598 tablet_bootstrap.cc:492] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f: No bootstrap required, opened a new log
I20260807 20:15:15.828923 13598 ts_tablet_manager.cc:1397] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260807 20:15:15.830602 13598 raft_consensus.cc:348] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2151dfe505d41609a85c8d16aa3a6bc" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 33639 } } peers { permanent_uuid: "e8c411e4002e4d1fbddeed25b307612f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44405 } } peers { permanent_uuid: "5be4493570b04012b6eb36882b77f4f3" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 36355 } }
I20260807 20:15:15.830734 13598 raft_consensus.cc:374] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:15.830766 13598 raft_consensus.cc:729] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8c411e4002e4d1fbddeed25b307612f, State: Initialized, Role: FOLLOWER
I20260807 20:15:15.830919 13598 consensus_queue.cc:260] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2151dfe505d41609a85c8d16aa3a6bc" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 33639 } } peers { permanent_uuid: "e8c411e4002e4d1fbddeed25b307612f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44405 } } peers { permanent_uuid: "5be4493570b04012b6eb36882b77f4f3" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 36355 } }
I20260807 20:15:15.831177 13598 ts_tablet_manager.cc:1428] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:15:15.831346 13320 heartbeater.cc:503] Master 127.23.200.254:37779 was elected leader, sending a full tablet report...
I20260807 20:15:15.832016 13599 tablet_bootstrap.cc:492] T 92a45f0c55ac44a6ba4341c3f30a2f79 P 5be4493570b04012b6eb36882b77f4f3: No bootstrap required, opened a new log
I20260807 20:15:15.832147 13599 ts_tablet_manager.cc:1397] T 92a45f0c55ac44a6ba4341c3f30a2f79 P 5be4493570b04012b6eb36882b77f4f3: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20260807 20:15:15.834230 13599 raft_consensus.cc:348] T 92a45f0c55ac44a6ba4341c3f30a2f79 P 5be4493570b04012b6eb36882b77f4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2151dfe505d41609a85c8d16aa3a6bc" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 33639 } } peers { permanent_uuid: "e8c411e4002e4d1fbddeed25b307612f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44405 } } peers { permanent_uuid: "5be4493570b04012b6eb36882b77f4f3" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 36355 } }
I20260807 20:15:15.834367 13599 raft_consensus.cc:374] T 92a45f0c55ac44a6ba4341c3f30a2f79 P 5be4493570b04012b6eb36882b77f4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:15.834398 13599 raft_consensus.cc:729] T 92a45f0c55ac44a6ba4341c3f30a2f79 P 5be4493570b04012b6eb36882b77f4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5be4493570b04012b6eb36882b77f4f3, State: Initialized, Role: FOLLOWER
I20260807 20:15:15.834512 13599 consensus_queue.cc:260] T 92a45f0c55ac44a6ba4341c3f30a2f79 P 5be4493570b04012b6eb36882b77f4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2151dfe505d41609a85c8d16aa3a6bc" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 33639 } } peers { permanent_uuid: "e8c411e4002e4d1fbddeed25b307612f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44405 } } peers { permanent_uuid: "5be4493570b04012b6eb36882b77f4f3" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 36355 } }
I20260807 20:15:15.834796 13108 heartbeater.cc:503] Master 127.23.200.254:37779 was elected leader, sending a full tablet report...
I20260807 20:15:15.834780 13599 ts_tablet_manager.cc:1428] T 92a45f0c55ac44a6ba4341c3f30a2f79 P 5be4493570b04012b6eb36882b77f4f3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:15:15.884557 13605 raft_consensus.cc:482] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:15:15.884866 13605 raft_consensus.cc:504] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2151dfe505d41609a85c8d16aa3a6bc" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 33639 } } peers { permanent_uuid: "e8c411e4002e4d1fbddeed25b307612f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44405 } } peers { permanent_uuid: "5be4493570b04012b6eb36882b77f4f3" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 36355 } }
I20260807 20:15:15.887745 13605 leader_election.cc:288] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2151dfe505d41609a85c8d16aa3a6bc (127.23.200.195:33639), 5be4493570b04012b6eb36882b77f4f3 (127.23.200.193:36355)
I20260807 20:15:15.891117 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:15.887670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:56837 (local address 127.23.200.193:36355)
0807 20:15:15.887712 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:15.887714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:15.888005 (+ 291us) server_negotiation.cc:407] Connection header received
0807 20:15:15.888098 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:15.888099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:15.888108 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:15.888134 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:15.888311 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.888958 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.890210 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.890743 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.890781 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:15.890788 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:15.891007 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 20:15:15.891043 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:15:15.891117 13593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:15.886070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:52311 (local address 127.23.200.195:33639)
0807 20:15:15.886114 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:15.886116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:15.886123 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:15:15.886151 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:15.886151 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:15.886161 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:15.886220 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:15.887806 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.888499 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.890094 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.890694 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.890742 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:15.890752 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:15.890997 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 20:15:15.891043 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:15:15.891367 13607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:15.885717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:33639 (local address 127.23.200.194:52311)
0807 20:15:15.886001 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:15.886013 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:15.886077 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:15.887620 (+ 1543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:15.887622 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:15.887624 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:15.887706 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:15.887709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.888586 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.888587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:15.889980 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:15.889983 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.890832 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.890834 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:15.890911 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:15.890914 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:15:15.891233 (+ 319us) client_negotiation.cc:240] Negotiation successful
0807 20:15:15.891283 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":190,"threads_started":1}
I20260807 20:15:15.891723 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:15.886418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:36355 (local address 127.23.200.194:56837)
0807 20:15:15.887944 (+ 1526us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:15.887954 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:15.888032 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:15.888189 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:15.888189 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:15.888191 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:15.888252 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:15.888254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.889054 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.889054 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:15.890117 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:15.890119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.890836 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.890837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:15.890932 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:15.890934 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:15:15.891613 (+ 679us) client_negotiation.cc:240] Negotiation successful
0807 20:15:15.891655 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1509,"mutex_wait_us":31,"thread_start_us":1086,"threads_started":1}
I20260807 20:15:15.891824 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92a45f0c55ac44a6ba4341c3f30a2f79" candidate_uuid: "e8c411e4002e4d1fbddeed25b307612f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2151dfe505d41609a85c8d16aa3a6bc" is_pre_election: true
I20260807 20:15:15.892027 13467 raft_consensus.cc:2454] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8c411e4002e4d1fbddeed25b307612f in term 0.
I20260807 20:15:15.892073 13022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92a45f0c55ac44a6ba4341c3f30a2f79" candidate_uuid: "e8c411e4002e4d1fbddeed25b307612f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5be4493570b04012b6eb36882b77f4f3" is_pre_election: true
I20260807 20:15:15.892230 13022 raft_consensus.cc:2454] T 92a45f0c55ac44a6ba4341c3f30a2f79 P 5be4493570b04012b6eb36882b77f4f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8c411e4002e4d1fbddeed25b307612f in term 0.
I20260807 20:15:15.892475 13152 leader_election.cc:302] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2151dfe505d41609a85c8d16aa3a6bc, e8c411e4002e4d1fbddeed25b307612f; no voters:
I20260807 20:15:15.892758 13605 raft_consensus.cc:2781] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:15:15.892824 13605 raft_consensus.cc:482] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:15:15.892846 13605 raft_consensus.cc:3037] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:15.893764 13605 raft_consensus.cc:504] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2151dfe505d41609a85c8d16aa3a6bc" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 33639 } } peers { permanent_uuid: "e8c411e4002e4d1fbddeed25b307612f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44405 } } peers { permanent_uuid: "5be4493570b04012b6eb36882b77f4f3" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 36355 } }
I20260807 20:15:15.893883 13605 leader_election.cc:288] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [CANDIDATE]: Term 1 election: Requested vote from peers b2151dfe505d41609a85c8d16aa3a6bc (127.23.200.195:33639), 5be4493570b04012b6eb36882b77f4f3 (127.23.200.193:36355)
I20260807 20:15:15.894060 13467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92a45f0c55ac44a6ba4341c3f30a2f79" candidate_uuid: "e8c411e4002e4d1fbddeed25b307612f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2151dfe505d41609a85c8d16aa3a6bc"
I20260807 20:15:15.894140 13467 raft_consensus.cc:3037] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:15.895059 13467 raft_consensus.cc:2454] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8c411e4002e4d1fbddeed25b307612f in term 1.
W20260807 20:15:15.895375 13111 tablet.cc:2369] T 92a45f0c55ac44a6ba4341c3f30a2f79 P 5be4493570b04012b6eb36882b77f4f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:15:15.895931 13022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92a45f0c55ac44a6ba4341c3f30a2f79" candidate_uuid: "e8c411e4002e4d1fbddeed25b307612f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5be4493570b04012b6eb36882b77f4f3"
I20260807 20:15:15.896010 13022 raft_consensus.cc:3037] T 92a45f0c55ac44a6ba4341c3f30a2f79 P 5be4493570b04012b6eb36882b77f4f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:15.896641 13022 raft_consensus.cc:2454] T 92a45f0c55ac44a6ba4341c3f30a2f79 P 5be4493570b04012b6eb36882b77f4f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8c411e4002e4d1fbddeed25b307612f in term 1.
I20260807 20:15:15.896761 13152 leader_election.cc:302] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2151dfe505d41609a85c8d16aa3a6bc, e8c411e4002e4d1fbddeed25b307612f; no voters:
I20260807 20:15:15.896847 13605 raft_consensus.cc:2781] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:15:15.896986 13605 raft_consensus.cc:686] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [term 1 LEADER]: Becoming Leader. State: Replica: e8c411e4002e4d1fbddeed25b307612f, State: Running, Role: LEADER
I20260807 20:15:15.897279 13605 consensus_queue.cc:237] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2151dfe505d41609a85c8d16aa3a6bc" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 33639 } } peers { permanent_uuid: "e8c411e4002e4d1fbddeed25b307612f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44405 } } peers { permanent_uuid: "5be4493570b04012b6eb36882b77f4f3" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 36355 } }
I20260807 20:15:15.898254 12783 catalog_manager.cc:5495] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f reported cstate change: term changed from 0 to 1, leader changed from <none> to e8c411e4002e4d1fbddeed25b307612f (127.23.200.194). New cstate: current_term: 1 leader_uuid: "e8c411e4002e4d1fbddeed25b307612f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2151dfe505d41609a85c8d16aa3a6bc" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 33639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8c411e4002e4d1fbddeed25b307612f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 44405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be4493570b04012b6eb36882b77f4f3" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 36355 } health_report { overall_health: UNKNOWN } } }
I20260807 20:15:15.923269 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:15.920818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49292 (local address 127.23.200.194:44405)
0807 20:15:15.920880 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:15.920882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:15.920912 (+ 30us) server_negotiation.cc:407] Connection header received
0807 20:15:15.921017 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:15.921017 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:15.921035 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:15.921122 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:15:15.921247 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.921723 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.922529 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:15.922741 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:15.922796 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:15.922946 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:15:15.923055 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:15:15.923128 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:15.923145 (+ 17us) server_negotiation.cc:299] Negotiation successful
0807 20:15:15.923167 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:15:15.925163 13022 raft_consensus.cc:1261] T 92a45f0c55ac44a6ba4341c3f30a2f79 P 5be4493570b04012b6eb36882b77f4f3 [term 1 FOLLOWER]: Refusing update from remote peer e8c411e4002e4d1fbddeed25b307612f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:15:15.925163 13467 raft_consensus.cc:1261] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc [term 1 FOLLOWER]: Refusing update from remote peer e8c411e4002e4d1fbddeed25b307612f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:15:15.925626 13605 consensus_queue.cc:1035] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be4493570b04012b6eb36882b77f4f3" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 36355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:15:15.925830 13612 consensus_queue.cc:1035] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2151dfe505d41609a85c8d16aa3a6bc" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 33639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:15:15.928447 13618 mvcc.cc:205] Tried to move back new op lower bound from 7316003700426649600 to 7316003700316192768. Current Snapshot: MvccSnapshot[applied={T|T < 7316003700426649600}]
I20260807 20:15:15.931376 12783 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:15:15.931468 12783 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:15:15.935266 12783 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 95fdd1c82f0a4b6493c26da3b5c0db89: Sending DeleteTablet for 3 replicas of tablet 92a45f0c55ac44a6ba4341c3f30a2f79
I20260807 20:15:15.935828 12985 tablet_service.cc:1514] Processing DeleteTablet for tablet 92a45f0c55ac44a6ba4341c3f30a2f79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:15:15 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:35416
I20260807 20:15:15.936174 13431 tablet_service.cc:1514] Processing DeleteTablet for tablet 92a45f0c55ac44a6ba4341c3f30a2f79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:15:15 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:60080
I20260807 20:15:15.936437 13217 tablet_service.cc:1514] Processing DeleteTablet for tablet 92a45f0c55ac44a6ba4341c3f30a2f79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:15:15 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:49282
I20260807 20:15:15.937060 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 12852
I20260807 20:15:15.952921 13620 mvcc.cc:205] Tried to move back new op lower bound from 7316003700426649600 to 7316003700316192768. Current Snapshot: MvccSnapshot[applied={T|T < 7316003700426649600}]
I20260807 20:15:15.957235 13630 tablet_replica.cc:330] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc: stopping tablet replica
I20260807 20:15:15.957471 13630 raft_consensus.cc:2229] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:15:15.957634 13634 tablet_replica.cc:330] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f: stopping tablet replica
I20260807 20:15:15.957831 13630 raft_consensus.cc:2258] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:15:15.957981 13634 raft_consensus.cc:2229] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:15:15.958202 13634 raft_consensus.cc:2258] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:15:15.958721 13630 ts_tablet_manager.cc:1905] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:15:15.960781 13634 ts_tablet_manager.cc:1905] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 20:15:15.962299 13152 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:36355 (error 108)
I20260807 20:15:15.962437 13634 ts_tablet_manager.cc:1918] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:15:15.962507 13634 log.cc:1196] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/92a45f0c55ac44a6ba4341c3f30a2f79
I20260807 20:15:15.962783 13630 ts_tablet_manager.cc:1918] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:15:15.962857 13630 log.cc:1196] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/92a45f0c55ac44a6ba4341c3f30a2f79
I20260807 20:15:15.963109 13630 ts_tablet_manager.cc:1939] T 92a45f0c55ac44a6ba4341c3f30a2f79 P b2151dfe505d41609a85c8d16aa3a6bc: Deleting consensus metadata
I20260807 20:15:15.963554 12762 catalog_manager.cc:4841] TS b2151dfe505d41609a85c8d16aa3a6bc (127.23.200.195:33639): tablet 92a45f0c55ac44a6ba4341c3f30a2f79 (table test-table [id=1347b072718045409306906685a729e4]) successfully deleted
I20260807 20:15:15.963898 13634 ts_tablet_manager.cc:1939] T 92a45f0c55ac44a6ba4341c3f30a2f79 P e8c411e4002e4d1fbddeed25b307612f: Deleting consensus metadata
I20260807 20:15:15.964351 12762 catalog_manager.cc:4841] TS e8c411e4002e4d1fbddeed25b307612f (127.23.200.194:44405): tablet 92a45f0c55ac44a6ba4341c3f30a2f79 (table test-table [id=1347b072718045409306906685a729e4]) successfully deleted
W20260807 20:15:15.968633 12762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:36355 (error 108)
W20260807 20:15:15.969134 13609 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:15:15.968555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:36355 (local address 127.23.200.194:60571)
0807 20:15:15.968621 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:15.968632 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:15.968677 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:15.969059 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:36355: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 20:15:15.969449 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 13114
I20260807 20:15:15.976191 13584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:15.976042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:36355 (local address 127.0.0.1:35418)
0807 20:15:15.976105 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:15.976152 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:36355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 20:15:15.977448 12762 catalog_manager.cc:4584] TS 5be4493570b04012b6eb36882b77f4f3 (127.23.200.193:36355): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 92a45f0c55ac44a6ba4341c3f30a2f79: Network error: Client connection negotiation failed: client connection to 127.23.200.193:36355: connect: Connection refused (error 111)
I20260807 20:15:15.985826 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 13326
I20260807 20:15:16.000123 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 12738
I20260807 20:15:16.009553 24355 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 20:15:16.353905 24355 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 20:15:16.354001 24355 mini_ranger.cc:66] Stopping Ranger...
I20260807 20:15:16.698360 24355 mini_ranger.cc:68] Stopped Ranger
2026-08-07 20:15:16.698 UTC [11221] LOG: received smart shutdown request
2026-08-07 20:15:16.703 UTC [11221] LOG: background worker "logical replication launcher" (PID 11235) exited with exit code 1
2026-08-07 20:15:16.703 UTC [11227] LOG: shutting down
2026-08-07 20:15:16.866 UTC [11221] LOG: database system is shut down
2026-08-07T20:15:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48160 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:15:17 dist-test-slave-2j7r krb5kdc[13648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:15:17 dist-test-slave-2j7r krb5kdc[13648](info): set up 2 sockets
Aug 07 20:15:17 dist-test-slave-2j7r krb5kdc[13648](info): commencing operation
krb5kdc: starting...
W20260807 20:15:18.116737 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:15:18 dist-test-slave-2j7r krb5kdc[13648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:15:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:15:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:15:18.244324 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:41181
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:41181,127.23.200.253:35903,127.23.200.252:35095
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:18.328071 13668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:18.328296 13668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:18.328326 13668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:18.329499 13668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:15:18.329538 13668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:18.329555 13668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:18.329571 13668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:15:18.329584 13668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:15:18.331072 13668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:41181,127.23.200.253:35903,127.23.200.252:35095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:41181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:18.331331 13668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:18.331550 13668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:18.336439 13677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:18.336452 13680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:18.336457 13676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:18.336550 13668 server_base.cc:1034] running on GCE node
I20260807 20:15:18.337182 13668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:18.337500 13668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:18.338642 13668 hybrid_clock.cc:648] HybridClock initialized: now 1786133718338628 us; error 29 us; skew 500 ppm
Aug 07 20:15:18 dist-test-slave-2j7r krb5kdc[13648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133718, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:18.341257 13668 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:15:18.341719 13668 webserver.cc:460] Webserver started at http://127.23.200.254:35747/ using document root <none> and password file <none>
I20260807 20:15:18.341946 13668 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:18.341996 13668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:18.342125 13668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:15:18.343209 13668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "81807e20903540fbbbbf7b7a9452c999"
format_stamp: "Formatted at 2026-08-07 20:15:18 on dist-test-slave-2j7r"
server_key: "047220e73f5c8236bb60410cde88baf9"
server_key_iv: "8c8dc82c0ae788802052e823a8f25e57"
server_key_version: "encryptionkey@0"
I20260807 20:15:18.343556 13668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "81807e20903540fbbbbf7b7a9452c999"
format_stamp: "Formatted at 2026-08-07 20:15:18 on dist-test-slave-2j7r"
server_key: "047220e73f5c8236bb60410cde88baf9"
server_key_iv: "8c8dc82c0ae788802052e823a8f25e57"
server_key_version: "encryptionkey@0"
I20260807 20:15:18.344811 13668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:15:18.345631 13692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:18.345830 13668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:15:18.345919 13668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "81807e20903540fbbbbf7b7a9452c999"
format_stamp: "Formatted at 2026-08-07 20:15:18 on dist-test-slave-2j7r"
server_key: "047220e73f5c8236bb60410cde88baf9"
server_key_iv: "8c8dc82c0ae788802052e823a8f25e57"
server_key_version: "encryptionkey@0"
I20260807 20:15:18.345997 13668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:18.356454 13668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:18.358430 13668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:18.358573 13668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:18.362699 13668 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:41181
I20260807 20:15:18.362746 13760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:41181 every 8 connection(s)
I20260807 20:15:18.363082 13668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:15:18.363952 13762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:18.365592 13762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:15:18.370210 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 13668
I20260807 20:15:18.370309 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:15:18.370519 24355 external_mini_cluster.cc:1428] Setting key 2e580acd1576a81c914a6b26f4a290d3
Aug 07 20:15:18 dist-test-slave-2j7r krb5kdc[13648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133718, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:18.375677 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.366331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:41181 (local address 127.0.0.1:54484)
0807 20:15:18.367113 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.367161 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:18.367615 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:18.367847 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:18.367849 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:18.368121 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:18.368286 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.368290 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.369352 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.369353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.370080 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.370083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.370623 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.370624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.370711 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.371190 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:18.371201 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:18.372670 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:18.374308 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.374314 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.374317 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.374505 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.374713 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.374713 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.374714 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.374801 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.375133 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:18.375138 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:18.375203 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 20:15:18.375284 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 20:15:18.375504 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":81,"thread_start_us":693,"threads_started":1}
I20260807 20:15:18.375680 13768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.367146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54484 (local address 127.23.200.254:41181)
0807 20:15:18.367279 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:18.367284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:18.367314 (+ 30us) server_negotiation.cc:407] Connection header received
0807 20:15:18.367738 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:18.367740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:18.367756 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:18.367834 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:18.368377 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.369266 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.370161 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.370514 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.370601 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.372756 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:18.372765 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:18.372765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:18.372772 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:18.374228 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.374576 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.374579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.374580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.374656 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.374858 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.374859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.374859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.375070 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:18.375144 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:18.375324 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 20:15:18.375430 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260807 20:15:18.376495 13762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:15:18.376837 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.376728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:35903 (local address 127.0.0.1:39154)
0807 20:15:18.376769 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.376812 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:35903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 20:15:18.376991 13696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.200.253:35903: connect: Connection refused (error 111)
I20260807 20:15:18.377521 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.377406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:35903 (local address 127.0.0.1:39158)
0807 20:15:18.377470 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.377500 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:35903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 20:15:18.377633 13762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.253:35903: Network error: Client connection negotiation failed: client connection to 127.23.200.253:35903: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.23.200.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.253.keytab.
Entry for principal kudu/127.23.200.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.253.keytab.
I20260807 20:15:18.396229 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.253:35903
--webserver_interface=127.23.200.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.253.keytab
--principal=kudu/127.23.200.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:41181,127.23.200.253:35903,127.23.200.252:35095
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:15:18.426882 13762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } attempt: 1
I20260807 20:15:18.427408 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.427258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:35903 (local address 127.0.0.1:39162)
0807 20:15:18.427321 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.427371 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:35903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 20:15:18.427729 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.427649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:35903 (local address 127.0.0.1:39172)
0807 20:15:18.427684 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.427710 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:35903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 20:15:18.427824 13762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.253:35903: Network error: Client connection negotiation failed: client connection to 127.23.200.253:35903: connect: Connection refused (error 111)
W20260807 20:15:18.479285 13773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:18.479486 13773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:18.479507 13773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:18.480610 13773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:15:18.480648 13773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:18.480660 13773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:18.480671 13773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:15:18.480683 13773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:15:18.482175 13773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:41181,127.23.200.253:35903,127.23.200.252:35095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.253.keytab
--principal=kudu/127.23.200.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.253:35903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.200.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:18.482422 13773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:18.482637 13773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:18.487493 13781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:18.487535 13780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:18.487545 13784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:18.487725 13773 server_base.cc:1034] running on GCE node
I20260807 20:15:18.488014 13773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:18.488301 13773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:18.489449 13773 hybrid_clock.cc:648] HybridClock initialized: now 1786133718489431 us; error 33 us; skew 500 ppm
Aug 07 20:15:18 dist-test-slave-2j7r krb5kdc[13648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133718, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:18.491825 13773 init.cc:377] Logged in from keytab as kudu/127.23.200.253@KRBTEST.COM (short username kudu)
I20260807 20:15:18.492264 13773 webserver.cc:460] Webserver started at http://127.23.200.253:35729/ using document root <none> and password file <none>
I20260807 20:15:18.492498 13773 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:18.492547 13773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:18.492671 13773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:15:18.493750 13773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/data/instance:
uuid: "bf3934ce4be74e9faa951ab9bbeb97c6"
format_stamp: "Formatted at 2026-08-07 20:15:18 on dist-test-slave-2j7r"
server_key: "db9d5e877eafa2a732164f63d7d3d481"
server_key_iv: "40f9f20760615204b449f2b370645e04"
server_key_version: "encryptionkey@0"
I20260807 20:15:18.494086 13773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/wal/instance:
uuid: "bf3934ce4be74e9faa951ab9bbeb97c6"
format_stamp: "Formatted at 2026-08-07 20:15:18 on dist-test-slave-2j7r"
server_key: "db9d5e877eafa2a732164f63d7d3d481"
server_key_iv: "40f9f20760615204b449f2b370645e04"
server_key_version: "encryptionkey@0"
I20260807 20:15:18.495281 13773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:15:18.495944 13793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:18.496002 13762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } attempt: 2
I20260807 20:15:18.496090 13773 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:15:18.496191 13773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/wal
uuid: "bf3934ce4be74e9faa951ab9bbeb97c6"
format_stamp: "Formatted at 2026-08-07 20:15:18 on dist-test-slave-2j7r"
server_key: "db9d5e877eafa2a732164f63d7d3d481"
server_key_iv: "40f9f20760615204b449f2b370645e04"
server_key_version: "encryptionkey@0"
I20260807 20:15:18.496259 13773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:18.496376 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.496241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:35903 (local address 127.0.0.1:39188)
0807 20:15:18.496300 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.496341 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:35903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 20:15:18.496683 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.496596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:35903 (local address 127.0.0.1:39196)
0807 20:15:18.496633 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.496662 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:35903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 20:15:18.496778 13762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.253:35903: Network error: Client connection negotiation failed: client connection to 127.23.200.253:35903: connect: Connection refused (error 111)
I20260807 20:15:18.512651 13773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:18.514525 13773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:18.514638 13773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:18.518494 13773 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.253:35903
I20260807 20:15:18.518540 13873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.253:35903 every 8 connection(s)
I20260807 20:15:18.518850 13773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/data/info.pb
I20260807 20:15:18.519680 13874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:18.521353 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 13773
I20260807 20:15:18.521481 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-1/wal/instance
I20260807 20:15:18.521271 13874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:15:18.521677 24355 external_mini_cluster.cc:1428] Setting key f1b774ad5485888d183c6549fdf9feab
Aug 07 20:15:18 dist-test-slave-2j7r krb5kdc[13648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133718, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.253@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.200.252@KRBTEST.COM; defaulting to no policy
I20260807 20:15:18.530256 13768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.522111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54492 (local address 127.23.200.254:41181)
0807 20:15:18.522169 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:18.522172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:18.522414 (+ 242us) server_negotiation.cc:407] Connection header received
0807 20:15:18.523005 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:18.523006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:18.523020 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:18.523058 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:18.523798 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.524278 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.525273 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.525492 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.525550 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.527591 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:18.527592 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:18.527593 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:18.527601 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:18.528920 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.529337 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.529339 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.529339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.529411 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.529632 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.529632 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.529633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.529747 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:18.529803 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:18.530071 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 20:15:18.530162 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 20:15:18.530556 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.522050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:41181 (local address 127.0.0.1:54492)
0807 20:15:18.522272 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.522322 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:18.522898 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:18.523135 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:18.523139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:18.523475 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:18.523702 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.523707 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.524364 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.524365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.525197 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.525200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.525563 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.525564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.525695 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.526177 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:18.526186 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:18.527502 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:18.528995 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.529003 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.529005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.529270 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.529467 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.529467 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.529468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.529577 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.529802 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:18.529810 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:18.529952 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 20:15:18.530044 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 20:15:18.530205 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":73,"threads_started":1}
I20260807 20:15:18.531103 13874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.252@KRBTEST.COM" created.
Aug 07 20:15:18 dist-test-slave-2j7r krb5kdc[13648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133718, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.253@KRBTEST.COM for kudu/127.23.200.253@KRBTEST.COM
I20260807 20:15:18.537955 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.531334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:35903 (local address 127.0.0.1:39212)
0807 20:15:18.531408 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.531419 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:18.531492 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:18.531692 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:18.531693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:18.531802 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:18.531853 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.531854 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.532715 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.532716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.533265 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.533267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.533827 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.533828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.533880 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.534238 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:18.534244 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:18.535503 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:18.536976 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.536977 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.536977 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.537170 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.537342 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.537342 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.537343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.537396 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.537639 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:18.537640 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:18.537678 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 20:15:18.537737 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 20:15:18.537834 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 20:15:18.538116 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.531402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39212 (local address 127.23.200.253:35903)
0807 20:15:18.531563 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:18.531567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:18.531582 (+ 15us) server_negotiation.cc:407] Connection header received
0807 20:15:18.531600 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:18.531601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:18.531612 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:18.531675 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:18.531907 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.532629 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.533336 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.533759 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.533831 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.535576 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:18.535585 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:18.535585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:18.535592 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:18.536907 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.537235 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.537238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.537239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.537289 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.537449 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.537450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.537450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.537582 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:18.537651 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:18.537766 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 20:15:18.537906 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260807 20:15:18.538734 13874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:15:18.539068 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.538959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:35095 (local address 127.0.0.1:43834)
0807 20:15:18.539000 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.539046 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:35095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 20:15:18.539201 13794 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.200.252:35095: connect: Connection refused (error 111)
I20260807 20:15:18.539618 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.539534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:35095 (local address 127.0.0.1:43844)
0807 20:15:18.539569 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.539593 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:35095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260807 20:15:18.539714 13874 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.252:35095: Network error: Client connection negotiation failed: client connection to 127.23.200.252:35095: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.252.keytab.
Entry for principal kudu/127.23.200.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.252.keytab.
I20260807 20:15:18.548508 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.252:35095
--webserver_interface=127.23.200.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.252.keytab
--principal=kudu/127.23.200.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:41181,127.23.200.253:35903,127.23.200.252:35095
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:15:18.587910 13762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } attempt: 3
I20260807 20:15:18.588907 13874 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } attempt: 1
I20260807 20:15:18.589258 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.589145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:35095 (local address 127.0.0.1:43848)
0807 20:15:18.589192 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.589225 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:35095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 20:15:18.589573 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.589483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:35095 (local address 127.0.0.1:43862)
0807 20:15:18.589523 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.589547 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:35095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 20:15:18.589651 13874 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.252:35095: Network error: Client connection negotiation failed: client connection to 127.23.200.252:35095: connect: Connection refused (error 111)
Aug 07 20:15:18 dist-test-slave-2j7r krb5kdc[13648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133718, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for kudu/127.23.200.253@KRBTEST.COM
I20260807 20:15:18.595144 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.588365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39224 (local address 127.23.200.253:35903)
0807 20:15:18.588428 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:18.588430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:18.588441 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:15:18.588471 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:18.588473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:18.588485 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:18.588563 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:18.588853 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.589676 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.590694 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.590906 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.590965 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.592838 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:18.592840 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:18.592840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:18.592847 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:18.594154 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.594461 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.594462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.594463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.594513 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.594673 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.594674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.594675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.594780 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:18.594837 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:18.594980 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 20:15:18.595051 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 20:15:18.595145 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.588270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:35903 (local address 127.0.0.1:39224)
0807 20:15:18.588325 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.588339 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:18.588391 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:18.588568 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:18.588569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:18.588721 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:18.588788 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.588789 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.589781 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.589783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.590603 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.590607 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.590984 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.590985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.591064 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.591461 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:18.591467 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:18.592759 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:18.594231 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.594233 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.594233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.594395 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.594565 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.594566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.594566 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.594625 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.594834 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:18.594835 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:18.594883 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 20:15:18.594949 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 20:15:18.595051 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 20:15:18.595532 13762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:15:18.595856 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.595766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:35095 (local address 127.0.0.1:43870)
0807 20:15:18.595808 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.595835 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:35095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 20:15:18.596168 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.596084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:35095 (local address 127.0.0.1:43884)
0807 20:15:18.596122 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.596145 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:35095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 20:15:18.596243 13762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.252:35095: Network error: Client connection negotiation failed: client connection to 127.23.200.252:35095: connect: Connection refused (error 111)
I20260807 20:15:18.627390 13762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } attempt: 1
I20260807 20:15:18.627936 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.627795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:35095 (local address 127.0.0.1:43894)
0807 20:15:18.627862 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.627906 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:35095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 20:15:18.628300 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.628210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:35095 (local address 127.0.0.1:43908)
0807 20:15:18.628256 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.628280 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:35095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 20:15:18.628443 13762 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.252:35095: Network error: Client connection negotiation failed: client connection to 127.23.200.252:35095: connect: Connection refused (error 111)
W20260807 20:15:18.633385 13887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:18.633574 13887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:18.633595 13887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:18.634699 13887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:15:18.634733 13887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:18.634747 13887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:18.634757 13887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:15:18.634774 13887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:15:18.636201 13887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:41181,127.23.200.253:35903,127.23.200.252:35095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.252.keytab
--principal=kudu/127.23.200.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.252:35095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.200.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:18.636402 13887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:18.636583 13887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:18.641299 13895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:18.641300 13900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:18.641315 13897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:18.641516 13887 server_base.cc:1034] running on GCE node
I20260807 20:15:18.641783 13887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:18.642061 13887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:18.643224 13887 hybrid_clock.cc:648] HybridClock initialized: now 1786133718643207 us; error 31 us; skew 500 ppm
Aug 07 20:15:18 dist-test-slave-2j7r krb5kdc[13648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133718, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:18.645668 13887 init.cc:377] Logged in from keytab as kudu/127.23.200.252@KRBTEST.COM (short username kudu)
I20260807 20:15:18.646090 13887 webserver.cc:460] Webserver started at http://127.23.200.252:45879/ using document root <none> and password file <none>
I20260807 20:15:18.646265 13887 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:18.646302 13887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:18.646394 13887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:15:18.647406 13887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/data/instance:
uuid: "e4b74129d2ba4a61bfba8b913d04ae2c"
format_stamp: "Formatted at 2026-08-07 20:15:18 on dist-test-slave-2j7r"
server_key: "b78f5d815cf0445ba014c90cfc3d41df"
server_key_iv: "0649b8158717162bdb882c1dc496697a"
server_key_version: "encryptionkey@0"
I20260807 20:15:18.647696 13887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/wal/instance:
uuid: "e4b74129d2ba4a61bfba8b913d04ae2c"
format_stamp: "Formatted at 2026-08-07 20:15:18 on dist-test-slave-2j7r"
server_key: "b78f5d815cf0445ba014c90cfc3d41df"
server_key_iv: "0649b8158717162bdb882c1dc496697a"
server_key_version: "encryptionkey@0"
I20260807 20:15:18.648872 13887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:15:18.649636 13913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:18.649783 13887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:18.649870 13887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/wal
uuid: "e4b74129d2ba4a61bfba8b913d04ae2c"
format_stamp: "Formatted at 2026-08-07 20:15:18 on dist-test-slave-2j7r"
server_key: "b78f5d815cf0445ba014c90cfc3d41df"
server_key_iv: "0649b8158717162bdb882c1dc496697a"
server_key_version: "encryptionkey@0"
I20260807 20:15:18.649946 13887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:18.657776 13874 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } attempt: 2
I20260807 20:15:18.658166 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.658050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:35095 (local address 127.0.0.1:43920)
0807 20:15:18.658102 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.658137 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:35095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 20:15:18.658397 13887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:18.658485 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.658398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:35095 (local address 127.0.0.1:43932)
0807 20:15:18.658440 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.658464 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:35095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 20:15:18.658567 13874 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.252:35095: Network error: Client connection negotiation failed: client connection to 127.23.200.252:35095: connect: Connection refused (error 111)
I20260807 20:15:18.660451 13887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:18.660583 13887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:18.664340 13887 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.252:35095
I20260807 20:15:18.664372 14001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.252:35095 every 8 connection(s)
I20260807 20:15:18.664706 13887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/data/info.pb
I20260807 20:15:18.665585 14004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:18.667104 14004 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:15:18 dist-test-slave-2j7r krb5kdc[13648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133718, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.252@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:18.674115 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 13887
I20260807 20:15:18.674225 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133562076833-24355-0/minicluster-data/master-2/wal/instance
I20260807 20:15:18.674451 24355 external_mini_cluster.cc:1428] Setting key 9da577ab76da6e718a3ee326d6176bf5
I20260807 20:15:18.675343 13768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.667783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54504 (local address 127.23.200.254:41181)
0807 20:15:18.667840 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:18.667842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:18.668024 (+ 182us) server_negotiation.cc:407] Connection header received
0807 20:15:18.668461 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:18.668462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:18.668471 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:18.668525 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:18.669069 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.669589 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.670503 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.670682 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.670724 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.672627 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:18.672629 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:18.672629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:18.672636 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:18.673951 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.674345 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.674346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.674347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.674411 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.674634 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.674636 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.674636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.674777 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:18.674843 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:18.675163 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 20:15:18.675257 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 20:15:18.675510 14006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.667750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:41181 (local address 127.0.0.1:54504)
0807 20:15:18.667906 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.667948 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:18.668372 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:18.668579 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:18.668582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:18.668818 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:18.669006 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.669009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.669663 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.669663 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.670441 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.670443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.670733 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.670733 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.670827 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.671219 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:18.671228 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:18.672532 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:18.674023 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.674030 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.674032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.674270 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.674472 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.674473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.674473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.674583 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.674836 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:18.674841 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:18.675000 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 20:15:18.675113 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 20:15:18.675269 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
I20260807 20:15:18.676021 14004 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:15:18 dist-test-slave-2j7r krb5kdc[13648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133718, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.252@KRBTEST.COM for kudu/127.23.200.253@KRBTEST.COM
Aug 07 20:15:18 dist-test-slave-2j7r krb5kdc[13648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.252@KRBTEST.COM
I20260807 20:15:18.682361 14006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.676253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:35903 (local address 127.0.0.1:39230)
0807 20:15:18.676299 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.676309 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:18.676350 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:18.676479 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:18.676480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:18.676618 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:18.676663 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.676664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.677445 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.677446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.678095 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.678097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.678515 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.678516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.678579 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.678860 (+ 281us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:18.678864 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:18.680025 (+ 1161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:18.681476 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.681478 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.681478 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.681654 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.681839 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.681839 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.681839 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.681883 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.682121 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:18.682122 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:18.682153 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 20:15:18.682193 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 20:15:18.682298 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 20:15:18.682390 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.676317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39230 (local address 127.23.200.253:35903)
0807 20:15:18.676361 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:18.676362 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:18.676370 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:15:18.676416 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:18.676416 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:18.676424 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:18.676466 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:18.676720 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.677372 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.678162 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.678448 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.678503 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.680105 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:18.680106 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:18.680107 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:18.680115 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:18.681406 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.681715 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.681716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.681716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.681776 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.681931 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.681932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.681933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.682057 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:18.682132 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:18.682232 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 20:15:18.682302 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:15:18.682713 14004 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:15:18.683846 14010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.674788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43936 (local address 127.23.200.252:35095)
0807 20:15:18.674946 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:18.674951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:18.675076 (+ 125us) server_negotiation.cc:407] Connection header received
0807 20:15:18.675125 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:18.675127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:18.675142 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:18.675218 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:18.675881 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.676811 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.677708 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.678165 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.678220 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.680711 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:18.680718 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:18.680718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:18.680725 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:18.682350 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.683057 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.683061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.683062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.683122 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.683310 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.683311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.683311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.683494 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:18.683621 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:18.683721 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 20:15:18.683778 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
Aug 07 20:15:18 dist-test-slave-2j7r krb5kdc[13648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133718, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.252@KRBTEST.COM for kudu/127.23.200.252@KRBTEST.COM
Aug 07 20:15:18 dist-test-slave-2j7r krb5kdc[13648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:18.689857 14006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.682934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:35095 (local address 127.0.0.1:43946)
0807 20:15:18.682972 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.682982 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:18.683017 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:18.683239 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:18.683240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:18.683370 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:18.683401 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.683403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.684228 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.684229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.684760 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.684762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.685221 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.685222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.685265 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.685610 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:18.685615 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:18.686974 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:18.688877 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.688878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.688878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.689067 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.689309 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.689309 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.689310 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.689361 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.689594 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:18.689595 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:18.689636 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 20:15:18.689706 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 20:15:18.689795 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260807 20:15:18.689996 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.682984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43946 (local address 127.23.200.252:35095)
0807 20:15:18.683137 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:18.683139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:18.683146 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:15:18.683162 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:18.683163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:18.683173 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:18.683239 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:18.683477 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.684145 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.684834 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.685155 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.685222 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.687046 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:18.687047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:18.687048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:18.687055 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:18.688803 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.689182 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.689183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.689184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.689242 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.689415 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.689416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.689416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.689539 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:18.689598 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:18.689726 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 20:15:18.689855 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260807 20:15:18.691588 14004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c: Bootstrap starting.
I20260807 20:15:18.692178 13768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.684811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54514 (local address 127.23.200.254:41181)
0807 20:15:18.684848 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:18.684850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:18.684858 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:15:18.684905 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:18.684905 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:18.684913 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:18.684954 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:18.685525 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.686192 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.687102 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.687362 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.687426 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.689552 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:18.689554 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:18.689554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:18.689559 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:18.691127 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.691441 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.691442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.691443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.691502 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.691660 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.691660 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.691661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.691783 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:18.691975 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:18.692014 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 20:15:18.692093 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:15:18.692350 14004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:18.692656 14004 log.cc:824] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:18.693496 14004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c: No bootstrap required, opened a new log
I20260807 20:15:18.695209 14004 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } }
I20260807 20:15:18.695363 14004 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:18.695407 14004 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4b74129d2ba4a61bfba8b913d04ae2c, State: Initialized, Role: FOLLOWER
I20260807 20:15:18.695541 14004 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } }
I20260807 20:15:18.695859 14004 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:15:18.695829 14013 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } } }
I20260807 20:15:18.695910 14013 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c [sys.catalog]: This master's current role is: FOLLOWER
Aug 07 20:15:18 dist-test-slave-2j7r krb5kdc[13648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.253@KRBTEST.COM
W20260807 20:15:18.697942 14029 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:15:18.698010 14029 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:15:18.697952 13887 master_runner.cc:186] Error getting master registration for 127.23.200.254:41181: OK, instance_id { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" instance_seqno: 1786133718361540 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 20:15:18.700234 13887 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 20:15:18.700541 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.692812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39240 (local address 127.23.200.253:35903)
0807 20:15:18.692854 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:18.692856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:18.692915 (+ 59us) server_negotiation.cc:407] Connection header received
0807 20:15:18.692976 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:18.692977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:18.692987 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:18.693050 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:18.693592 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.694058 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.694917 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.695083 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.695122 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.697630 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:18.697632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:18.697632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:18.697641 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:18.699272 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.699666 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.699668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.699668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.699727 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.699902 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.699902 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.699903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.700028 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:18.700206 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:18.700386 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 20:15:18.700456 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:15:18.702487 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:15:18.703570 13762 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } attempt: 2
Aug 07 20:15:18 dist-test-slave-2j7r krb5kdc[13648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133718, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for kudu/127.23.200.252@KRBTEST.COM
I20260807 20:15:18.712641 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.703965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:35095 (local address 127.0.0.1:43960)
0807 20:15:18.704003 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.704012 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:18.704054 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:18.704205 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:18.704206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:18.704376 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:18.704413 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.704415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.705305 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.705306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.706058 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.706062 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.706552 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.706553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.706605 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.707018 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:18.707023 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:18.708491 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:18.710650 (+ 2159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.710651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.710652 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.711586 (+ 934us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.711785 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.711785 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.711786 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.711842 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.712112 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:18.712113 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:18.712156 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 20:15:18.712212 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 20:15:18.712537 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 20:15:18.712694 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.703887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43956 (local address 127.23.200.252:35095)
0807 20:15:18.703923 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:18.703924 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:18.705116 (+ 1192us) server_negotiation.cc:407] Connection header received
0807 20:15:18.705199 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:18.705200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:18.705208 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:18.705234 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:18.705875 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.706559 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.707449 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.707751 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.707792 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.709144 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:18.709145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:18.709146 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:18.709153 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:18.711463 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.711840 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.711841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.711842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.711899 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.712098 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.712099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.712099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.712242 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:18.712344 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:18.712532 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 20:15:18.712601 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:15:18.713362 14010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.703991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43960 (local address 127.23.200.252:35095)
0807 20:15:18.704018 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:18.704020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:18.704040 (+ 20us) server_negotiation.cc:407] Connection header received
0807 20:15:18.704106 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:18.704106 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:18.704117 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:18.704162 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:18.704504 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.705226 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.706140 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.706452 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.706503 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.708579 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:18.708581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:18.708581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:18.708588 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:18.710260 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.711659 (+ 1399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.711661 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.711661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.711721 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.711914 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.711915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.711916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.712037 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:18.712090 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:18.713197 (+ 1107us) server_negotiation.cc:299] Negotiation successful
0807 20:15:18.713277 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 20:15:18.714834 13768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.703766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54526 (local address 127.23.200.254:41181)
0807 20:15:18.703809 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:18.703811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:18.704206 (+ 395us) server_negotiation.cc:407] Connection header received
0807 20:15:18.704288 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:18.704289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:18.704297 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:18.704325 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:18.705009 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.705725 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.707007 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.707295 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.707339 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.709308 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:18.709309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:18.709310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:18.709316 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:18.711457 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.713941 (+ 2484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.713942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.713943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.713996 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.714277 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.714278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.714278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.714404 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:18.714483 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:18.714687 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 20:15:18.714760 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:15:18.715679 13762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999: Bootstrap starting.
I20260807 20:15:18.716389 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.704163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39256 (local address 127.23.200.253:35903)
0807 20:15:18.704200 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:18.704201 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:18.708196 (+ 3995us) server_negotiation.cc:407] Connection header received
0807 20:15:18.709436 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:18.709437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:18.709450 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:18.709501 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:18.710038 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.710694 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.711654 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.711979 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.712040 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.713629 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:18.713630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:18.713631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:18.713639 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:18.715257 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.715600 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.715601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.715601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.715658 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.715853 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.715853 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.715854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.715991 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:18.716085 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:18.716220 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 20:15:18.716297 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:15:18.716450 13762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:18.716760 13762 log.cc:824] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:18.717625 13762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999: No bootstrap required, opened a new log
I20260807 20:15:18.719005 13762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } }
I20260807 20:15:18.719118 13762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:18.719139 13762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81807e20903540fbbbbf7b7a9452c999, State: Initialized, Role: FOLLOWER
I20260807 20:15:18.719260 13762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } }
I20260807 20:15:18.719514 14041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } } }
I20260807 20:15:18.719583 13762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:15:18.719592 14041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 20:15:18.721253 14064 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:15:18.721335 14064 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:15:18.722301 13668 master_runner.cc:186] Error getting master registration for 127.23.200.253:35903: OK, instance_id { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" instance_seqno: 1786133718517372 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 20:15:18.724520 13668 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 20:15:18.749707 13874 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } attempt: 3
Aug 07 20:15:18 dist-test-slave-2j7r krb5kdc[13648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133718, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.253@KRBTEST.COM for kudu/127.23.200.252@KRBTEST.COM
I20260807 20:15:18.757691 14010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.750059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43976 (local address 127.23.200.252:35095)
0807 20:15:18.750106 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:18.750108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:18.750116 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:15:18.750138 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:18.750139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:18.750150 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:18.750212 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:18.750537 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.751304 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.752281 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.752610 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.752670 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.755002 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:18.755004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:18.755004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:18.755012 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:18.756538 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.756920 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.756921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.756921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.756980 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:18.757177 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:18.757178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:18.757178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:18.757311 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:18.757374 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:18.757520 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 20:15:18.757596 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:15:18.757709 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:18.749991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:35095 (local address 127.0.0.1:43976)
0807 20:15:18.750043 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:18.750054 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:18.750091 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:18.750222 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:18.750223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:18.750414 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:18.750467 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.750469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.751386 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.751387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.752207 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:18.752209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:18.752691 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:18.752692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:18.752754 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:18.753343 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:18.753349 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:18.754913 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:18.756631 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.756632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.756633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.756851 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.757040 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:18.757041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:18.757041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:18.757126 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:18.757370 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:18.757371 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:18.757418 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 20:15:18.757492 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 20:15:18.757589 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 20:15:18.760051 13874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6: Bootstrap starting.
I20260807 20:15:18.760828 13874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:18.761207 13874 log.cc:824] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:18.762154 13874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6: No bootstrap required, opened a new log
I20260807 20:15:18.764304 13874 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } }
I20260807 20:15:18.764463 13874 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:18.764549 13874 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf3934ce4be74e9faa951ab9bbeb97c6, State: Initialized, Role: FOLLOWER
I20260807 20:15:18.764701 13874 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } }
I20260807 20:15:18.765131 13874 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 20:15:18.765174 13802 tablet.cc:2369] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:15:18.765046 14068 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } } }
I20260807 20:15:18.765259 14068 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 20:15:18.767289 14084 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:15:18.767341 14084 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:15:18.769832 13773 master_runner.cc:263] No leader master found from master bf3934ce4be74e9faa951ab9bbeb97c6
I20260807 20:15:18.769891 13773 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 20:15:18.779820 14068 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:15:18.779964 14068 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } }
I20260807 20:15:18.780190 14068 leader_election.cc:288] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81807e20903540fbbbbf7b7a9452c999 (127.23.200.254:41181), e4b74129d2ba4a61bfba8b913d04ae2c (127.23.200.252:35095)
I20260807 20:15:18.780594 13959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" is_pre_election: true
I20260807 20:15:18.780764 13959 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf3934ce4be74e9faa951ab9bbeb97c6 in term 0.
I20260807 20:15:18.781123 13794 leader_election.cc:302] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf3934ce4be74e9faa951ab9bbeb97c6, e4b74129d2ba4a61bfba8b913d04ae2c; no voters:
I20260807 20:15:18.781348 14068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:15:18.781410 14068 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:15:18.781435 14068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:18.782671 14068 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } }
I20260807 20:15:18.782785 14068 leader_election.cc:288] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [CANDIDATE]: Term 1 election: Requested vote from peers 81807e20903540fbbbbf7b7a9452c999 (127.23.200.254:41181), e4b74129d2ba4a61bfba8b913d04ae2c (127.23.200.252:35095)
I20260807 20:15:18.783226 13728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81807e20903540fbbbbf7b7a9452c999" is_pre_election: true
I20260807 20:15:18.783382 13728 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf3934ce4be74e9faa951ab9bbeb97c6 in term 0.
I20260807 20:15:18.783236 13727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81807e20903540fbbbbf7b7a9452c999"
I20260807 20:15:18.783547 13727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:18.784382 13727 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf3934ce4be74e9faa951ab9bbeb97c6 in term 1.
I20260807 20:15:18.784790 13959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c"
I20260807 20:15:18.784873 13959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:18.785775 13959 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf3934ce4be74e9faa951ab9bbeb97c6 in term 1.
I20260807 20:15:18.786053 13797 leader_election.cc:302] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81807e20903540fbbbbf7b7a9452c999, bf3934ce4be74e9faa951ab9bbeb97c6; no voters:
I20260807 20:15:18.786322 14068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:15:18.786510 14068 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [term 1 LEADER]: Becoming Leader. State: Replica: bf3934ce4be74e9faa951ab9bbeb97c6, State: Running, Role: LEADER
I20260807 20:15:18.786607 14068 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } }
I20260807 20:15:18.787053 14068 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf3934ce4be74e9faa951ab9bbeb97c6. Latest consensus state: current_term: 1 leader_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } } }
I20260807 20:15:18.787127 14068 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:18.787946 14088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:15:18.788197 14088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:15:18.789950 13959 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c [term 1 FOLLOWER]: Refusing update from remote peer bf3934ce4be74e9faa951ab9bbeb97c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:15:18.790367 14068 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:15:18.791843 14013 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf3934ce4be74e9faa951ab9bbeb97c6. Latest consensus state: current_term: 1 leader_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } } }
I20260807 20:15:18.791960 14013 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:15:18.792404 13727 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999 [term 1 FOLLOWER]: Refusing update from remote peer bf3934ce4be74e9faa951ab9bbeb97c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:15:18.792719 14068 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:15:18.794950 14041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf3934ce4be74e9faa951ab9bbeb97c6. Latest consensus state: current_term: 1 leader_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } } }
I20260807 20:15:18.795037 14041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:15:18.796447 14086 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } } }
I20260807 20:15:18.796540 14086 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:18.797015 14041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } } }
I20260807 20:15:18.797113 14041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:15:18.797420 14090 mvcc.cc:205] Tried to move back new op lower bound from 7316003712161001472 to 7316003712150581248. Current Snapshot: MvccSnapshot[applied={T|T < 7316003712161001472}]
I20260807 20:15:18.797780 14094 mvcc.cc:205] Tried to move back new op lower bound from 7316003712161001472 to 7316003712150581248. Current Snapshot: MvccSnapshot[applied={T|T < 7316003712161001472}]
I20260807 20:15:18.798729 14068 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } } }
I20260807 20:15:18.798807 14068 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6 [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:18.799263 14013 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81807e20903540fbbbbf7b7a9452c999" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 41181 } } peers { permanent_uuid: "bf3934ce4be74e9faa951ab9bbeb97c6" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 35903 } } peers { permanent_uuid: "e4b74129d2ba4a61bfba8b913d04ae2c" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 35095 } } }
I20260807 20:15:18.799345 14013 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:15:18.800329 14091 mvcc.cc:205] Tried to move back new op lower bound from 7316003712161001472 to 7316003712150581248. Current Snapshot: MvccSnapshot[applied={T|T < 7316003712161001472}]
I20260807 20:15:18.800930 14088 catalog_manager.cc:1347] Generated new cluster ID: ecc62699324846ae93b3b9d5baea5002
I20260807 20:15:18.800976 14088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:15:18.812206 14088 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:15:18.813036 14088 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:15:18.826504 14088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf3934ce4be74e9faa951ab9bbeb97c6: Generated new TSK 0
I20260807 20:15:18.826818 14088 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:15:19.698478 14029 catalog_manager.cc:1259] Loaded cluster ID: ecc62699324846ae93b3b9d5baea5002
I20260807 20:15:19.698553 14029 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c: loading cluster ID for follower catalog manager: success
I20260807 20:15:19.699503 14029 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c: acquiring CA information for follower catalog manager: success
I20260807 20:15:19.699820 14029 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e4b74129d2ba4a61bfba8b913d04ae2c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 20:15:19.721807 14064 catalog_manager.cc:1259] Loaded cluster ID: ecc62699324846ae93b3b9d5baea5002
I20260807 20:15:19.721884 14064 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999: loading cluster ID for follower catalog manager: success
I20260807 20:15:19.722661 14064 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999: acquiring CA information for follower catalog manager: success
I20260807 20:15:19.722930 14064 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 81807e20903540fbbbbf7b7a9452c999: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 20:15:19.895051 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 13668
I20260807 20:15:19.908159 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 13773
I20260807 20:15:19.919592 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 13887
2026-08-07T20:15:19Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2968 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:15:19 dist-test-slave-2j7r krb5kdc[14116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:15:19 dist-test-slave-2j7r krb5kdc[14116](info): set up 2 sockets
Aug 07 20:15:19 dist-test-slave-2j7r krb5kdc[14116](info): commencing operation
krb5kdc: starting...
W20260807 20:15:21.104816 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:15:21 dist-test-slave-2j7r krb5kdc[14116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:15:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:15:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:15:21.232447 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:42803
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:42803
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:21.316303 14134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:21.316515 14134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:21.316545 14134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:21.317726 14134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:15:21.317776 14134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:21.317797 14134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:21.317812 14134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:15:21.317828 14134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:15:21.319314 14134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:42803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:42803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:21.319561 14134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:21.319775 14134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:21.324733 14143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:21.324764 14144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:21.324765 14134 server_base.cc:1034] running on GCE node
W20260807 20:15:21.324733 14146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:21.325178 14134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:21.325464 14134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:21.326629 14134 hybrid_clock.cc:648] HybridClock initialized: now 1786133721326614 us; error 31 us; skew 500 ppm
Aug 07 20:15:21 dist-test-slave-2j7r krb5kdc[14116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133721, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:21.329243 14134 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:15:21.329653 14134 webserver.cc:460] Webserver started at http://127.23.200.254:44349/ using document root <none> and password file <none>
I20260807 20:15:21.329839 14134 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:21.329878 14134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:21.329984 14134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:15:21.330984 14134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "0a40182de99f4ef6aa73f14bb111477a"
format_stamp: "Formatted at 2026-08-07 20:15:21 on dist-test-slave-2j7r"
server_key: "614bf3077e2c382ab5b41a52e1da53e5"
server_key_iv: "232529cfdcfb8b052ce58cd0a2dc9a99"
server_key_version: "encryptionkey@0"
I20260807 20:15:21.331272 14134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "0a40182de99f4ef6aa73f14bb111477a"
format_stamp: "Formatted at 2026-08-07 20:15:21 on dist-test-slave-2j7r"
server_key: "614bf3077e2c382ab5b41a52e1da53e5"
server_key_iv: "232529cfdcfb8b052ce58cd0a2dc9a99"
server_key_version: "encryptionkey@0"
I20260807 20:15:21.332463 14134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:15:21.333153 14154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:21.333302 14134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:21.333391 14134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "0a40182de99f4ef6aa73f14bb111477a"
format_stamp: "Formatted at 2026-08-07 20:15:21 on dist-test-slave-2j7r"
server_key: "614bf3077e2c382ab5b41a52e1da53e5"
server_key_iv: "232529cfdcfb8b052ce58cd0a2dc9a99"
server_key_version: "encryptionkey@0"
I20260807 20:15:21.333452 14134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:21.350467 14134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:21.352491 14134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:21.352607 14134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:21.356494 14134 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:42803
I20260807 20:15:21.356559 14227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:42803 every 8 connection(s)
I20260807 20:15:21.356892 14134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:15:21.357858 14228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:21.358184 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 14134
I20260807 20:15:21.358271 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:15:21.358466 24355 external_mini_cluster.cc:1428] Setting key 4b61d92d540612009f9e3078cbf079cf
I20260807 20:15:21.360769 14228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a: Bootstrap starting.
I20260807 20:15:21.361517 14228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:21.361807 14228 log.cc:824] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:21.362514 14228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a: No bootstrap required, opened a new log
I20260807 20:15:21.364568 14228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a40182de99f4ef6aa73f14bb111477a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42803 } }
I20260807 20:15:21.364711 14228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:21.364742 14228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a40182de99f4ef6aa73f14bb111477a, State: Initialized, Role: FOLLOWER
I20260807 20:15:21.364873 14228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a40182de99f4ef6aa73f14bb111477a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42803 } }
I20260807 20:15:21.364951 14228 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 07 20:15:21 dist-test-slave-2j7r krb5kdc[14116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:21.364995 14228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:15:21.365051 14228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:21.365764 14228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a40182de99f4ef6aa73f14bb111477a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42803 } }
I20260807 20:15:21.365871 14228 leader_election.cc:302] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a40182de99f4ef6aa73f14bb111477a; no voters:
I20260807 20:15:21.366109 14228 leader_election.cc:288] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:15:21.366155 14235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:15:21.366297 14235 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [term 1 LEADER]: Becoming Leader. State: Replica: 0a40182de99f4ef6aa73f14bb111477a, State: Running, Role: LEADER
I20260807 20:15:21.366439 14228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:15:21.366675 14235 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a40182de99f4ef6aa73f14bb111477a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42803 } }
I20260807 20:15:21.367033 14235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a40182de99f4ef6aa73f14bb111477a. Latest consensus state: current_term: 1 leader_uuid: "0a40182de99f4ef6aa73f14bb111477a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a40182de99f4ef6aa73f14bb111477a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42803 } } }
I20260807 20:15:21.367245 14235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:21.367180 14238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a40182de99f4ef6aa73f14bb111477a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a40182de99f4ef6aa73f14bb111477a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42803 } } }
I20260807 20:15:21.367622 14238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:21.367980 14250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:15:21.368661 14250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:15:21.369585 14233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:21.358948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45746 (local address 127.23.200.254:42803)
0807 20:15:21.359156 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:21.359169 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:21.359206 (+ 37us) server_negotiation.cc:407] Connection header received
0807 20:15:21.359712 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:21.359722 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:21.359783 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:21.359945 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:21.360497 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:21.361808 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:21.362544 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:21.363139 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:21.363257 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:21.365551 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:21.365556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:21.365556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:21.365567 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:21.367524 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:21.367975 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:21.367978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:21.367979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:21.368118 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:21.368335 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:21.368336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:21.368336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:21.368651 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:21.368811 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:21.369307 (+ 496us) server_negotiation.cc:299] Negotiation successful
0807 20:15:21.369436 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":81,"threads_started":1}
I20260807 20:15:21.370584 14250 catalog_manager.cc:1347] Generated new cluster ID: 92d486c72c7a46baabacdce79f628de1
I20260807 20:15:21.370662 14250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:15:21.370949 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:15:21.380609 14250 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:15:21.381206 14250 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:15:21.388551 14250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a40182de99f4ef6aa73f14bb111477a: Generated new TSK 0
I20260807 20:15:21.388736 14250 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:15:21.425071 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 14134
2026-08-07T20:15:21Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1499 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:15:21 dist-test-slave-2j7r krb5kdc[14270](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:15:21 dist-test-slave-2j7r krb5kdc[14270](info): set up 2 sockets
Aug 07 20:15:21 dist-test-slave-2j7r krb5kdc[14270](info): commencing operation
krb5kdc: starting...
W20260807 20:15:22.582036 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:15:22 dist-test-slave-2j7r krb5kdc[14270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:15:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:15:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:15:22.708184 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:45999
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:45999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:22.791730 14289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:22.791953 14289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:22.791985 14289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:22.793195 14289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:15:22.793241 14289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:22.793267 14289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:22.793285 14289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:15:22.793309 14289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:15:22.794864 14289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:45999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:45999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:22.795132 14289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:22.795356 14289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:22.800464 14300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:22.800585 14306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:22.800540 14289 server_base.cc:1034] running on GCE node
W20260807 20:15:22.800527 14301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:22.800951 14289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:22.801293 14289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:22.802475 14289 hybrid_clock.cc:648] HybridClock initialized: now 1786133722802458 us; error 30 us; skew 500 ppm
Aug 07 20:15:22 dist-test-slave-2j7r krb5kdc[14270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133722, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:22.805212 14289 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:15:22.805725 14289 webserver.cc:460] Webserver started at http://127.23.200.254:36971/ using document root <none> and password file <none>
I20260807 20:15:22.805936 14289 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:22.805992 14289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:22.806126 14289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:15:22.807184 14289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "2253a25b07cb4d8b876b6b8c2571eaff"
format_stamp: "Formatted at 2026-08-07 20:15:22 on dist-test-slave-2j7r"
server_key: "63343f36d69f242819cfb44bdc296c31"
server_key_iv: "03c08f933ccad33029c51f84342496a2"
server_key_version: "encryptionkey@0"
I20260807 20:15:22.807528 14289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "2253a25b07cb4d8b876b6b8c2571eaff"
format_stamp: "Formatted at 2026-08-07 20:15:22 on dist-test-slave-2j7r"
server_key: "63343f36d69f242819cfb44bdc296c31"
server_key_iv: "03c08f933ccad33029c51f84342496a2"
server_key_version: "encryptionkey@0"
I20260807 20:15:22.808804 14289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:15:22.809643 14316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:22.809795 14289 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:22.809885 14289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "2253a25b07cb4d8b876b6b8c2571eaff"
format_stamp: "Formatted at 2026-08-07 20:15:22 on dist-test-slave-2j7r"
server_key: "63343f36d69f242819cfb44bdc296c31"
server_key_iv: "03c08f933ccad33029c51f84342496a2"
server_key_version: "encryptionkey@0"
I20260807 20:15:22.809954 14289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:22.831808 14289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:22.834038 14289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:22.834184 14289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:22.838243 14289 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:45999
I20260807 20:15:22.838294 14397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:45999 every 8 connection(s)
I20260807 20:15:22.838658 14289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:15:22.839687 14398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:22.842271 14398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff: Bootstrap starting.
I20260807 20:15:22.842882 14398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:22.843127 14398 log.cc:824] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:22.843817 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 14289
I20260807 20:15:22.843953 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:15:22.843976 14398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff: No bootstrap required, opened a new log
I20260807 20:15:22.844178 24355 external_mini_cluster.cc:1428] Setting key 491e151cfcb50e0233e59e61f603461b
I20260807 20:15:22.845960 14398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } }
I20260807 20:15:22.846110 14398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:22.846135 14398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2253a25b07cb4d8b876b6b8c2571eaff, State: Initialized, Role: FOLLOWER
I20260807 20:15:22.846294 14398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } }
I20260807 20:15:22.846385 14398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:15:22.846421 14398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:15:22.846474 14398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:22.847460 14398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } }
I20260807 20:15:22.847551 14398 leader_election.cc:302] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2253a25b07cb4d8b876b6b8c2571eaff; no voters:
I20260807 20:15:22.847702 14398 leader_election.cc:288] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:15:22.847743 14404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:15:22.847959 14398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:15:22.847963 14404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 1 LEADER]: Becoming Leader. State: Replica: 2253a25b07cb4d8b876b6b8c2571eaff, State: Running, Role: LEADER
I20260807 20:15:22.848191 14404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } }
I20260807 20:15:22.848675 14405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } } }
I20260807 20:15:22.848760 14405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:22.848991 14407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2253a25b07cb4d8b876b6b8c2571eaff. Latest consensus state: current_term: 1 leader_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } } }
I20260807 20:15:22.849128 14407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:22.849299 14419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:15:22.849882 14419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:15:22.851563 14419 catalog_manager.cc:1347] Generated new cluster ID: ce5e9117a1934553a4f7d37a6c4daaee
I20260807 20:15:22.851615 14419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:15:22 dist-test-slave-2j7r krb5kdc[14270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:22.855494 14403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:22.844739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34794 (local address 127.23.200.254:45999)
0807 20:15:22.844943 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:22.844953 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:22.844981 (+ 28us) server_negotiation.cc:407] Connection header received
0807 20:15:22.845966 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:22.845975 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:22.846035 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:22.846189 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:22.846932 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:22.848039 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:22.849200 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:22.849760 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:22.849869 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:22.852362 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:22.852366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:22.852367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:22.852380 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:22.853936 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:22.854257 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:22.854261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:22.854261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:22.854365 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:22.854538 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:22.854539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:22.854539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:22.854777 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:22.854934 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:22.855005 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 20:15:22.855139 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
I20260807 20:15:22.876151 14419 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:15:22.876700 14419 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:15:22.880921 14419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff: Generated new TSK 0
I20260807 20:15:22.881135 14419 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:15:22.913908 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45999
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:23.014037 14438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:23.014240 14438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:23.014269 14438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:23.015400 14438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:23.015443 14438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:23.015467 14438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:15:23.016960 14438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:23.017230 14438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:23.017470 14438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:23.022606 14456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:23.022652 14452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:23.022768 14449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:23.022735 14438 server_base.cc:1034] running on GCE node
I20260807 20:15:23.023031 14438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:23.023272 14438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:23.024506 14438 hybrid_clock.cc:648] HybridClock initialized: now 1786133723024491 us; error 30 us; skew 500 ppm
Aug 07 20:15:23 dist-test-slave-2j7r krb5kdc[14270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:23.027035 14438 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:15:23.027479 14438 webserver.cc:460] Webserver started at http://127.23.200.193:36785/ using document root <none> and password file <none>
I20260807 20:15:23.027696 14438 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:23.027745 14438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:23.027878 14438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:15:23.028898 14438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "80664e6c8320444a8cef8c7da0479c34"
format_stamp: "Formatted at 2026-08-07 20:15:23 on dist-test-slave-2j7r"
server_key: "1b618b0016259f48d037fcc56b18101f"
server_key_iv: "d29e91e30019de92f8f2b543fb8fce2f"
server_key_version: "encryptionkey@0"
I20260807 20:15:23.029247 14438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "80664e6c8320444a8cef8c7da0479c34"
format_stamp: "Formatted at 2026-08-07 20:15:23 on dist-test-slave-2j7r"
server_key: "1b618b0016259f48d037fcc56b18101f"
server_key_iv: "d29e91e30019de92f8f2b543fb8fce2f"
server_key_version: "encryptionkey@0"
I20260807 20:15:23.030539 14438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:15:23.031215 14472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:23.031363 14438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:15:23.031455 14438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "80664e6c8320444a8cef8c7da0479c34"
format_stamp: "Formatted at 2026-08-07 20:15:23 on dist-test-slave-2j7r"
server_key: "1b618b0016259f48d037fcc56b18101f"
server_key_iv: "d29e91e30019de92f8f2b543fb8fce2f"
server_key_version: "encryptionkey@0"
I20260807 20:15:23.031514 14438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:23.043643 14438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:23.045642 14438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:23.045780 14438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:23.046020 14438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:23.046319 14438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:15:23.046360 14438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:23.046398 14438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:23.046423 14438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:23.051947 14438 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:41929
I20260807 20:15:23.051992 14637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:41929 every 8 connection(s)
I20260807 20:15:23.052302 14438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
Aug 07 20:15:23 dist-test-slave-2j7r krb5kdc[14270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:23.060240 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 14438
I20260807 20:15:23.060346 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:15:23.060546 24355 external_mini_cluster.cc:1428] Setting key 314ba12a3c0fb562fa1dd6ef41323a35
I20260807 20:15:23.061636 14403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.052989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:35449 (local address 127.23.200.254:45999)
0807 20:15:23.053049 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:23.053051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:23.053349 (+ 298us) server_negotiation.cc:407] Connection header received
0807 20:15:23.053856 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:23.053857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:23.053876 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:23.053929 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:23.054886 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.055404 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.056390 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.056607 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.056663 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.058744 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:23.058747 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:23.058747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:23.058759 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:23.060100 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:23.060581 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:23.060583 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:23.060583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:23.060657 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:23.060924 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:23.060924 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:23.060925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:23.061044 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:23.061137 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:23.061465 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 20:15:23.061562 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:15:23.061848 14641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.052952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:35449)
0807 20:15:23.053134 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:23.053187 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:23.053759 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:23.054000 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:23.054003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:23.054592 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:23.054788 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.054793 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.055512 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.055513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.056307 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.056309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.056678 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.056679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.056796 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.057316 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:23.057325 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:23.058647 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:23.060213 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:23.060222 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:23.060225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:23.060507 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:23.060748 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:23.060749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:23.060750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:23.060868 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:23.061143 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:23.061150 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:23.061278 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 20:15:23.061446 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 20:15:23.061620 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":69,"threads_started":1}
I20260807 20:15:23.062335 14638 heartbeater.cc:347] Connected to a master server at 127.23.200.254:45999
I20260807 20:15:23.062443 14638 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:23.062644 14638 heartbeater.cc:511] Master 127.23.200.254:45999 requested a full tablet report, sending...
I20260807 20:15:23.063268 14336 ts_manager.cc:194] Registered new tserver with Master: 80664e6c8320444a8cef8c7da0479c34 (127.23.200.193:41929)
I20260807 20:15:23.064134 14336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:35449
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:15:23.087415 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45999
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:23.171974 14645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:23.172192 14645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:23.172217 14645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:23.173341 14645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:23.173384 14645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:23.173411 14645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:15:23.174862 14645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:23.175066 14645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:23.175276 14645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:23.180507 14654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:23.180608 14653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:23.180516 14656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:23.180532 14645 server_base.cc:1034] running on GCE node
I20260807 20:15:23.180948 14645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:23.181200 14645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:23.182356 14645 hybrid_clock.cc:648] HybridClock initialized: now 1786133723182341 us; error 32 us; skew 500 ppm
Aug 07 20:15:23 dist-test-slave-2j7r krb5kdc[14270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:23.184927 14645 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:15:23.185395 14645 webserver.cc:460] Webserver started at http://127.23.200.194:38813/ using document root <none> and password file <none>
I20260807 20:15:23.185627 14645 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:23.185674 14645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:23.185802 14645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:15:23.186820 14645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "3e3cfea450f6430ca1eb4000d703af9f"
format_stamp: "Formatted at 2026-08-07 20:15:23 on dist-test-slave-2j7r"
server_key: "d82e5afb4dd2b070cf5ff6c76f250a54"
server_key_iv: "ccc14b27761254297975ead5b8ea9e6c"
server_key_version: "encryptionkey@0"
I20260807 20:15:23.187148 14645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "3e3cfea450f6430ca1eb4000d703af9f"
format_stamp: "Formatted at 2026-08-07 20:15:23 on dist-test-slave-2j7r"
server_key: "d82e5afb4dd2b070cf5ff6c76f250a54"
server_key_iv: "ccc14b27761254297975ead5b8ea9e6c"
server_key_version: "encryptionkey@0"
I20260807 20:15:23.188421 14645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:15:23.189261 14667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:23.189409 14645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:15:23.189503 14645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "3e3cfea450f6430ca1eb4000d703af9f"
format_stamp: "Formatted at 2026-08-07 20:15:23 on dist-test-slave-2j7r"
server_key: "d82e5afb4dd2b070cf5ff6c76f250a54"
server_key_iv: "ccc14b27761254297975ead5b8ea9e6c"
server_key_version: "encryptionkey@0"
I20260807 20:15:23.189570 14645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:23.206135 14645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:23.207968 14645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:23.208109 14645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:23.208338 14645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:23.208655 14645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:15:23.208689 14645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:23.208719 14645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:23.208737 14645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:23.214470 14645 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:38955
I20260807 20:15:23.214526 14822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:38955 every 8 connection(s)
I20260807 20:15:23.214833 14645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
Aug 07 20:15:23 dist-test-slave-2j7r krb5kdc[14270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:23.222893 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 14645
I20260807 20:15:23.223016 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:15:23.223201 24355 external_mini_cluster.cc:1428] Setting key f20470d167f89a5ae575dced450f207e
I20260807 20:15:23.223898 14403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.215591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:37495 (local address 127.23.200.254:45999)
0807 20:15:23.215679 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:23.215681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:23.215895 (+ 214us) server_negotiation.cc:407] Connection header received
0807 20:15:23.216397 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:23.216398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:23.216412 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:23.216460 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:23.217503 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.218014 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.219010 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.219211 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.219266 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.221338 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:23.221340 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:23.221340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:23.221348 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:23.222585 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:23.222961 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:23.222963 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:23.222963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:23.223025 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:23.223246 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:23.223247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:23.223247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:23.223381 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:23.223447 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:23.223709 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 20:15:23.223799 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 20:15:23.224061 14831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.215559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:37495)
0807 20:15:23.215765 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:23.215804 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:23.216301 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:23.216567 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:23.216570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:23.217200 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:23.217410 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.217415 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.218108 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.218109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.218927 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.218929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.219286 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.219287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.219410 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.219870 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:23.219880 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:23.221244 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:23.222661 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:23.222669 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:23.222671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:23.222889 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:23.223078 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:23.223079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:23.223079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:23.223189 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:23.223444 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:23.223452 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:23.223590 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 20:15:23.223680 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 20:15:23.223837 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":97,"threads_started":1}
I20260807 20:15:23.224474 14823 heartbeater.cc:347] Connected to a master server at 127.23.200.254:45999
I20260807 20:15:23.224567 14823 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:23.224745 14823 heartbeater.cc:511] Master 127.23.200.254:45999 requested a full tablet report, sending...
I20260807 20:15:23.225150 14336 ts_manager.cc:194] Registered new tserver with Master: 3e3cfea450f6430ca1eb4000d703af9f (127.23.200.194:38955)
I20260807 20:15:23.225605 14336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:37495
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:15:23.249440 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45999
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:23.332300 14836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:23.332520 14836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:23.332549 14836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:23.333742 14836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:23.333783 14836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:23.333818 14836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:15:23.335393 14836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:23.335642 14836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:23.335867 14836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:23.340996 14842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:23.340960 14846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:23.340976 14841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:23.341007 14836 server_base.cc:1034] running on GCE node
I20260807 20:15:23.341426 14836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:23.341651 14836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:23.342808 14836 hybrid_clock.cc:648] HybridClock initialized: now 1786133723342793 us; error 29 us; skew 500 ppm
Aug 07 20:15:23 dist-test-slave-2j7r krb5kdc[14270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:23.345347 14836 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:15:23.345750 14836 webserver.cc:460] Webserver started at http://127.23.200.195:39379/ using document root <none> and password file <none>
I20260807 20:15:23.345963 14836 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:23.346004 14836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:23.346093 14836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:15:23.347124 14836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "9b03878eb76d4c4d9ad14bb3dd221eea"
format_stamp: "Formatted at 2026-08-07 20:15:23 on dist-test-slave-2j7r"
server_key: "a386e610d1087ba33857f2b2bb86c7be"
server_key_iv: "906d76d890f2dee9820477d491e24dbd"
server_key_version: "encryptionkey@0"
I20260807 20:15:23.347422 14836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "9b03878eb76d4c4d9ad14bb3dd221eea"
format_stamp: "Formatted at 2026-08-07 20:15:23 on dist-test-slave-2j7r"
server_key: "a386e610d1087ba33857f2b2bb86c7be"
server_key_iv: "906d76d890f2dee9820477d491e24dbd"
server_key_version: "encryptionkey@0"
I20260807 20:15:23.348691 14836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 20:15:23.349609 14861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:23.349797 14836 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:15:23.349896 14836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "9b03878eb76d4c4d9ad14bb3dd221eea"
format_stamp: "Formatted at 2026-08-07 20:15:23 on dist-test-slave-2j7r"
server_key: "a386e610d1087ba33857f2b2bb86c7be"
server_key_iv: "906d76d890f2dee9820477d491e24dbd"
server_key_version: "encryptionkey@0"
I20260807 20:15:23.349968 14836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:23.372627 14836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:23.374701 14836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:23.374827 14836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:23.375083 14836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:23.375406 14836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:15:23.375442 14836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:23.375473 14836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:23.375496 14836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:23.381341 14836 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:41701
I20260807 20:15:23.381522 15033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:41701 every 8 connection(s)
I20260807 20:15:23.381709 14836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:15:23.384909 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 14836
I20260807 20:15:23.384992 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:15:23.385191 24355 external_mini_cluster.cc:1428] Setting key 89accc3afb225189127dd89891aced94
Aug 07 20:15:23 dist-test-slave-2j7r krb5kdc[14270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:23.390965 14403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.382389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:42689 (local address 127.23.200.254:45999)
0807 20:15:23.382436 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:23.382437 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:23.382638 (+ 201us) server_negotiation.cc:407] Connection header received
0807 20:15:23.383168 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:23.383169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:23.383184 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:23.383236 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:23.384198 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.384757 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.386106 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.386306 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.386360 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.388501 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:23.388503 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:23.388503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:23.388511 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:23.389680 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:23.390069 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:23.390070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:23.390071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:23.390131 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:23.390355 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:23.390356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:23.390357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:23.390470 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:23.390526 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:23.390787 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 20:15:23.390876 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:15:23.391137 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.382355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:42689)
0807 20:15:23.382517 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:23.382555 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:23.383067 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:23.383317 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:23.383321 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:23.383903 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:23.384102 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.384108 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.384844 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.384846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.386044 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.386048 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.386374 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.386375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.386517 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.386982 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:23.386991 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:23.388410 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:23.389773 (+ 1363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:23.389780 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:23.389783 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:23.390001 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:23.390186 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:23.390187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:23.390187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:23.390301 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:23.390525 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:23.390531 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:23.390669 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 20:15:23.390761 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 20:15:23.390916 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260807 20:15:23.391594 15035 heartbeater.cc:347] Connected to a master server at 127.23.200.254:45999
I20260807 20:15:23.391680 15035 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:23.391875 15035 heartbeater.cc:511] Master 127.23.200.254:45999 requested a full tablet report, sending...
I20260807 20:15:23.392243 14336 ts_manager.cc:194] Registered new tserver with Master: 9b03878eb76d4c4d9ad14bb3dd221eea (127.23.200.195:41701)
I20260807 20:15:23.392655 14336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:42689
I20260807 20:15:23.396889 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:15:23.403756 14403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.398327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34802 (local address 127.23.200.254:45999)
0807 20:15:23.398393 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:23.398395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:23.398405 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:15:23.398470 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:23.398471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:23.398479 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:23.398530 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:23.399063 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.399550 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.400257 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.400451 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.400510 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.401659 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:23.401661 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:23.401661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:23.401668 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:23.402818 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:23.403115 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:23.403115 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:23.403115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:23.403161 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:23.403327 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:23.403328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:23.403328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:23.403463 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:23.403535 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:23.403636 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 20:15:23.403691 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 20:15:23.405160 14336 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:15:23.406808 14336 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:15:23.412943 15056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.409008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:41701 (local address 127.0.0.1:47488)
0807 20:15:23.409459 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:23.409471 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:23.409532 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:23.409756 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:23.409759 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:23.409762 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:23.409835 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.409836 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.410772 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.410773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.411781 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.411783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.412623 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.412624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.412689 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.412699 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 20:15:23.412757 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 20:15:23.412815 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":20,"thread_start_us":96,"threads_started":1}
I20260807 20:15:23.412949 15060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.409254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47488 (local address 127.23.200.195:41701)
0807 20:15:23.409436 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:23.409441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:23.409591 (+ 150us) server_negotiation.cc:407] Connection header received
0807 20:15:23.409639 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:23.409641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:23.409658 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:23.409737 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:23.409924 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.410686 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.411864 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.412464 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.412549 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.412593 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:23.412786 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 20:15:23.412829 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260807 20:15:23.413362 15055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.409009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:38955 (local address 127.0.0.1:47500)
0807 20:15:23.409170 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:23.409212 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:23.409281 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:23.409963 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:23.409964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:23.409965 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:23.410012 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.410013 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.411331 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.411332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.412221 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.412223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.413158 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.413159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.413222 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.413226 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:15:23.413253 (+ 27us) client_negotiation.cc:240] Negotiation successful
0807 20:15:23.413288 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"spinlock_wait_cycles":384,"thread_start_us":100,"threads_started":1}
I20260807 20:15:23.413697 15059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.409121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47500 (local address 127.23.200.194:38955)
0807 20:15:23.409613 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:23.409617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:23.409626 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:15:23.409660 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:23.409662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:23.409865 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:23.410071 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:23.410154 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.411231 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.412293 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.413033 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.413474 (+ 441us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.413549 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:23.413574 (+ 25us) server_negotiation.cc:299] Negotiation successful
0807 20:15:23.413618 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":94,"threads_started":1}
I20260807 20:15:23.413822 14934 tablet_service.cc:1467] Processing CreateTablet for tablet 2429853b38b742f4a396964f26893df4 (DEFAULT_TABLE table=test-table [id=8e4c7eb5b97a4060a21006a633174d39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:15:23.414171 14934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2429853b38b742f4a396964f26893df4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:23.414448 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.409285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:41929 (local address 127.0.0.1:57250)
0807 20:15:23.409456 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:23.409464 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:23.409504 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:23.409997 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:23.409998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:23.410001 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:23.410037 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.410038 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.412310 (+ 2272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.412310 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.413396 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.413399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.414210 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.414212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.414273 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.414278 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:15:23.414347 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 20:15:23.414385 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":126,"threads_started":1}
I20260807 20:15:23.414474 15064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.409566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57250 (local address 127.23.200.193:41929)
0807 20:15:23.409769 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:23.409773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:23.409782 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:15:23.409818 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:23.409820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:23.409835 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:23.409916 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:23.410114 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.411225 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.413484 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.414134 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.414195 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.414233 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:23.414365 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 20:15:23.414410 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":88,"threads_started":1}
I20260807 20:15:23.416703 15066 tablet_bootstrap.cc:492] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea: Bootstrap starting.
I20260807 20:15:23.417439 15066 tablet_bootstrap.cc:654] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:23.417495 14734 tablet_service.cc:1467] Processing CreateTablet for tablet 2429853b38b742f4a396964f26893df4 (DEFAULT_TABLE table=test-table [id=8e4c7eb5b97a4060a21006a633174d39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:15:23.417661 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 2429853b38b742f4a396964f26893df4 (DEFAULT_TABLE table=test-table [id=8e4c7eb5b97a4060a21006a633174d39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:15:23.417825 14734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2429853b38b742f4a396964f26893df4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:23.417989 15066 log.cc:824] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:23.418017 14534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2429853b38b742f4a396964f26893df4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:23.418802 15066 tablet_bootstrap.cc:492] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea: No bootstrap required, opened a new log
I20260807 20:15:23.418880 15066 ts_tablet_manager.cc:1397] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:15:23.420630 15071 tablet_bootstrap.cc:492] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f: Bootstrap starting.
I20260807 20:15:23.420814 15066 raft_consensus.cc:348] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } }
I20260807 20:15:23.420987 15066 raft_consensus.cc:374] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:23.421065 15066 raft_consensus.cc:729] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b03878eb76d4c4d9ad14bb3dd221eea, State: Initialized, Role: FOLLOWER
I20260807 20:15:23.421216 15066 consensus_queue.cc:260] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } }
I20260807 20:15:23.421427 15071 tablet_bootstrap.cc:654] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:23.421512 15035 heartbeater.cc:503] Master 127.23.200.254:45999 was elected leader, sending a full tablet report...
I20260807 20:15:23.421496 15066 ts_tablet_manager.cc:1428] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:15:23.421975 15071 log.cc:824] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:23.422300 15072 tablet_bootstrap.cc:492] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34: Bootstrap starting.
I20260807 20:15:23.422719 15071 tablet_bootstrap.cc:492] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f: No bootstrap required, opened a new log
I20260807 20:15:23.422789 15071 ts_tablet_manager.cc:1397] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:15:23.422869 15072 tablet_bootstrap.cc:654] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:23.423323 15072 log.cc:824] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:23.423950 15072 tablet_bootstrap.cc:492] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34: No bootstrap required, opened a new log
I20260807 20:15:23.424006 15072 ts_tablet_manager.cc:1397] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:15:23.424293 15071 raft_consensus.cc:348] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } }
I20260807 20:15:23.424433 15071 raft_consensus.cc:374] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:23.424454 15071 raft_consensus.cc:729] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3cfea450f6430ca1eb4000d703af9f, State: Initialized, Role: FOLLOWER
I20260807 20:15:23.424561 15071 consensus_queue.cc:260] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } }
I20260807 20:15:23.424804 15071 ts_tablet_manager.cc:1428] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:15:23.424839 14823 heartbeater.cc:503] Master 127.23.200.254:45999 was elected leader, sending a full tablet report...
I20260807 20:15:23.425787 15072 raft_consensus.cc:348] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } }
I20260807 20:15:23.425921 15072 raft_consensus.cc:374] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:23.425966 15072 raft_consensus.cc:729] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80664e6c8320444a8cef8c7da0479c34, State: Initialized, Role: FOLLOWER
I20260807 20:15:23.426075 15072 consensus_queue.cc:260] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } }
I20260807 20:15:23.426301 15072 ts_tablet_manager.cc:1428] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:15:23.426359 14638 heartbeater.cc:503] Master 127.23.200.254:45999 was elected leader, sending a full tablet report...
W20260807 20:15:23.465426 14827 tablet.cc:2369] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:15:23.517328 15073 raft_consensus.cc:482] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:15:23.517521 15073 raft_consensus.cc:504] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } }
I20260807 20:15:23.517951 15073 leader_election.cc:288] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3cfea450f6430ca1eb4000d703af9f (127.23.200.194:38955), 80664e6c8320444a8cef8c7da0479c34 (127.23.200.193:41929)
I20260807 20:15:23.521018 15064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.518479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:33307 (local address 127.23.200.193:41929)
0807 20:15:23.518531 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:23.518533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:23.518651 (+ 118us) server_negotiation.cc:407] Connection header received
0807 20:15:23.518681 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:23.518682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:23.518691 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:23.518755 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:23.518922 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.519403 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.520322 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.520689 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.520752 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.520762 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:23.520904 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 20:15:23.520948 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:15:23.521039 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.518179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:38955 (local address 127.23.200.195:44663)
0807 20:15:23.518236 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:23.518248 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:23.518302 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:23.518622 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:23.518623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:23.518625 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:23.518706 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.518708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.519378 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.519379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.520171 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.520174 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.520770 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.520771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.520864 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.520867 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:15:23.520922 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 20:15:23.520959 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 20:15:23.521039 15059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.518341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:44663 (local address 127.23.200.194:38955)
0807 20:15:23.518429 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:23.518431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:23.518440 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:15:23.518472 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:23.518473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:23.518482 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:23.518595 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:23.518776 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.519305 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.520262 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.520690 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.520757 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.520766 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:23.520940 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 20:15:23.520979 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 20:15:23.521046 15079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.518371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:41929 (local address 127.23.200.195:33307)
0807 20:15:23.518566 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:23.518576 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:23.518635 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:23.518774 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:23.518774 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:23.518776 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:23.518841 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.518842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.519479 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.519480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.520227 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.520228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.520775 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.520776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.520837 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.520841 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:15:23.520888 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 20:15:23.520923 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":121,"threads_started":1}
I20260807 20:15:23.521485 14764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2429853b38b742f4a396964f26893df4" candidate_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3cfea450f6430ca1eb4000d703af9f" is_pre_election: true
I20260807 20:15:23.521485 14564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2429853b38b742f4a396964f26893df4" candidate_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80664e6c8320444a8cef8c7da0479c34" is_pre_election: true
I20260807 20:15:23.521652 14564 raft_consensus.cc:2454] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b03878eb76d4c4d9ad14bb3dd221eea in term 0.
I20260807 20:15:23.521652 14764 raft_consensus.cc:2454] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b03878eb76d4c4d9ad14bb3dd221eea in term 0.
I20260807 20:15:23.521900 14866 leader_election.cc:302] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80664e6c8320444a8cef8c7da0479c34, 9b03878eb76d4c4d9ad14bb3dd221eea; no voters:
I20260807 20:15:23.522033 15073 raft_consensus.cc:2781] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:15:23.522091 15073 raft_consensus.cc:482] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:15:23.522119 15073 raft_consensus.cc:3037] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:23.522797 15073 raft_consensus.cc:504] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } }
I20260807 20:15:23.522914 15073 leader_election.cc:288] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [CANDIDATE]: Term 1 election: Requested vote from peers 3e3cfea450f6430ca1eb4000d703af9f (127.23.200.194:38955), 80664e6c8320444a8cef8c7da0479c34 (127.23.200.193:41929)
I20260807 20:15:23.523101 14764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2429853b38b742f4a396964f26893df4" candidate_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3cfea450f6430ca1eb4000d703af9f"
I20260807 20:15:23.523146 14564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2429853b38b742f4a396964f26893df4" candidate_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80664e6c8320444a8cef8c7da0479c34"
I20260807 20:15:23.523181 14764 raft_consensus.cc:3037] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:23.523240 14564 raft_consensus.cc:3037] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:23.523895 14764 raft_consensus.cc:2454] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b03878eb76d4c4d9ad14bb3dd221eea in term 1.
I20260807 20:15:23.524044 14564 raft_consensus.cc:2454] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b03878eb76d4c4d9ad14bb3dd221eea in term 1.
I20260807 20:15:23.524047 14866 leader_election.cc:302] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3cfea450f6430ca1eb4000d703af9f, 9b03878eb76d4c4d9ad14bb3dd221eea; no voters:
I20260807 20:15:23.524194 15073 raft_consensus.cc:2781] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:15:23.524349 15073 raft_consensus.cc:686] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 1 LEADER]: Becoming Leader. State: Replica: 9b03878eb76d4c4d9ad14bb3dd221eea, State: Running, Role: LEADER
I20260807 20:15:23.524430 15073 consensus_queue.cc:237] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } }
I20260807 20:15:23.525314 14336 catalog_manager.cc:5495] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b03878eb76d4c4d9ad14bb3dd221eea (127.23.200.195). New cstate: current_term: 1 leader_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } health_report { overall_health: HEALTHY } } }
I20260807 20:15:23.531040 15060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.529112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47498 (local address 127.23.200.195:41701)
0807 20:15:23.529155 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:23.529156 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:23.529163 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:15:23.529190 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:23.529190 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:23.529206 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:23.529261 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:15:23.529353 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.529791 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.530458 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.530635 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.530676 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.530769 (+ 93us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:15:23.530862 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:15:23.530928 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:23.530939 (+ 11us) server_negotiation.cc:299] Negotiation successful
0807 20:15:23.530954 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:15:23.532676 14564 raft_consensus.cc:1261] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34 [term 1 FOLLOWER]: Refusing update from remote peer 9b03878eb76d4c4d9ad14bb3dd221eea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:15:23.532685 14764 raft_consensus.cc:1261] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f [term 1 FOLLOWER]: Refusing update from remote peer 9b03878eb76d4c4d9ad14bb3dd221eea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:15:23.532987 15073 consensus_queue.cc:1035] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [LEADER]: Connected to new peer: Peer: permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:15:23.533128 15081 consensus_queue.cc:1035] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:15:23.535532 15088 mvcc.cc:205] Tried to move back new op lower bound from 7316003731587493888 to 7316003731556712448. Current Snapshot: MvccSnapshot[applied={T|T < 7316003731587493888}]
I20260807 20:15:23.537472 14336 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:15:23.537559 14336 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:15:23.538244 14336 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff: Sending DeleteTablet for 3 replicas of tablet 2429853b38b742f4a396964f26893df4
I20260807 20:15:23.538609 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 14438
I20260807 20:15:23.538668 14934 tablet_service.cc:1514] Processing DeleteTablet for tablet 2429853b38b742f4a396964f26893df4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:15:23 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:47488
I20260807 20:15:23.538692 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet 2429853b38b742f4a396964f26893df4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:15:23 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:57250
I20260807 20:15:23.541445 14734 tablet_service.cc:1514] Processing DeleteTablet for tablet 2429853b38b742f4a396964f26893df4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:15:23 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:47500
I20260807 20:15:23.544656 15102 tablet_replica.cc:330] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f: stopping tablet replica
I20260807 20:15:23.544909 15102 raft_consensus.cc:2229] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:15:23.545176 15102 raft_consensus.cc:2258] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:15:23.545984 15101 tablet_replica.cc:330] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea: stopping tablet replica
I20260807 20:15:23.546114 15101 raft_consensus.cc:2229] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:15:23.546228 15101 raft_consensus.cc:2258] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:15:23.546672 15101 ts_tablet_manager.cc:1905] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:15:23.548621 15101 ts_tablet_manager.cc:1918] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:15:23.548760 15101 log.cc:1196] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/2429853b38b742f4a396964f26893df4
I20260807 20:15:23.549059 15101 ts_tablet_manager.cc:1939] T 2429853b38b742f4a396964f26893df4 P 9b03878eb76d4c4d9ad14bb3dd221eea: Deleting consensus metadata
I20260807 20:15:23.549682 14321 catalog_manager.cc:4841] TS 9b03878eb76d4c4d9ad14bb3dd221eea (127.23.200.195:41701): tablet 2429853b38b742f4a396964f26893df4 (table test-table [id=8e4c7eb5b97a4060a21006a633174d39]) successfully deleted
I20260807 20:15:23.556146 15102 ts_tablet_manager.cc:1905] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:15:23.558128 15102 ts_tablet_manager.cc:1918] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:15:23.558233 15102 log.cc:1196] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/2429853b38b742f4a396964f26893df4
I20260807 20:15:23.558561 15102 ts_tablet_manager.cc:1939] T 2429853b38b742f4a396964f26893df4 P 3e3cfea450f6430ca1eb4000d703af9f: Deleting consensus metadata
I20260807 20:15:23.559098 14320 catalog_manager.cc:4841] TS 3e3cfea450f6430ca1eb4000d703af9f (127.23.200.194:38955): tablet 2429853b38b742f4a396964f26893df4 (table test-table [id=8e4c7eb5b97a4060a21006a633174d39]) successfully deleted
W20260807 20:15:23.560214 14320 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:41929 (error 108)
I20260807 20:15:23.560793 15062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.560660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:41929 (local address 127.0.0.1:57256)
0807 20:15:23.560714 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:23.560757 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:41929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 20:15:23.560823 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 14645
W20260807 20:15:23.577687 14320 catalog_manager.cc:4584] TS 80664e6c8320444a8cef8c7da0479c34 (127.23.200.193:41929): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2429853b38b742f4a396964f26893df4: Network error: Client connection negotiation failed: client connection to 127.23.200.193:41929: connect: Connection refused (error 111)
I20260807 20:15:23.587131 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 14836
I20260807 20:15:23.601461 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 14289
I20260807 20:15:23.611802 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:45999
--webserver_interface=127.23.200.254
--webserver_port=36971
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:45999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:23.694171 15105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:23.694388 15105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:23.694419 15105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:23.695818 15105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:15:23.695886 15105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:23.695910 15105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:23.695926 15105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:15:23.695942 15105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:15:23.697683 15105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:45999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:45999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=36971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:23.697937 15105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:23.698153 15105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:23.707930 15119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:23.708014 15117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:23.707996 15105 server_base.cc:1034] running on GCE node
W20260807 20:15:23.708021 15116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:23.708526 15105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:23.708837 15105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:23.709995 15105 hybrid_clock.cc:648] HybridClock initialized: now 1786133723709979 us; error 29 us; skew 500 ppm
Aug 07 20:15:23 dist-test-slave-2j7r krb5kdc[14270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:23.712682 15105 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:15:23.713140 15105 webserver.cc:460] Webserver started at http://127.23.200.254:36971/ using document root <none> and password file <none>
I20260807 20:15:23.713382 15105 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:23.713447 15105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:23.714893 15105 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 20:15:23.715535 15128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:23.715706 15105 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:15:23.715798 15105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "2253a25b07cb4d8b876b6b8c2571eaff"
format_stamp: "Formatted at 2026-08-07 20:15:22 on dist-test-slave-2j7r"
server_key: "63343f36d69f242819cfb44bdc296c31"
server_key_iv: "03c08f933ccad33029c51f84342496a2"
server_key_version: "encryptionkey@0"
I20260807 20:15:23.716180 15105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:23.726511 15105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:23.728312 15105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:23.728444 15105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:23.732249 15105 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:45999
I20260807 20:15:23.732291 15207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:45999 every 8 connection(s)
I20260807 20:15:23.732654 15105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:15:23.734315 15208 sys_catalog.cc:263] Verifying existing consensus state
I20260807 20:15:23.734980 15208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff: Bootstrap starting.
I20260807 20:15:23.737064 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 15105
I20260807 20:15:23.737445 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:41929
--local_ip_for_outbound_sockets=127.23.200.193
--tserver_master_addrs=127.23.200.254:45999
--webserver_port=36785
--webserver_interface=127.23.200.193
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:15:23.743825 15208 log.cc:824] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:23.745770 15208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:15:23.746074 15208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff: Bootstrap complete.
I20260807 20:15:23.748203 15208 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } }
I20260807 20:15:23.748422 15208 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2253a25b07cb4d8b876b6b8c2571eaff, State: Initialized, Role: FOLLOWER
I20260807 20:15:23.748566 15208 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } }
I20260807 20:15:23.748627 15208 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:15:23.748660 15208 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:15:23.748708 15208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 1 FOLLOWER]: Advancing to term 2
I20260807 20:15:23.749734 15208 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } }
I20260807 20:15:23.749825 15208 leader_election.cc:302] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2253a25b07cb4d8b876b6b8c2571eaff; no voters:
I20260807 20:15:23.749989 15208 leader_election.cc:288] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 20:15:23.750047 15215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 2 FOLLOWER]: Leader election won for term 2
I20260807 20:15:23.750164 15215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 2 LEADER]: Becoming Leader. State: Replica: 2253a25b07cb4d8b876b6b8c2571eaff, State: Running, Role: LEADER
I20260807 20:15:23.750262 15208 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:15:23.750320 15215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } }
I20260807 20:15:23.750617 15215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2253a25b07cb4d8b876b6b8c2571eaff. Latest consensus state: current_term: 2 leader_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } } }
I20260807 20:15:23.750695 15215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:23.750833 15218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } } }
I20260807 20:15:23.750890 15218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:23.751155 15227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:15:23.751935 15227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:15:23.752039 15227 catalog_manager.cc:1259] Loaded cluster ID: ce5e9117a1934553a4f7d37a6c4daaee
I20260807 20:15:23.752071 15227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:15:23.752369 15227 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 20:15:23.756939 15227 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 20:15:23.759356 15227 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:15:23.759588 15227 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff: Loaded TSK: 0
I20260807 20:15:23.759830 15227 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 20:15:23.823586 15212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:23.823796 15212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:23.823829 15212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:23.824980 15212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:23.825023 15212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:23.825058 15212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:15:23.826568 15212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:41929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=36785
--tserver_master_addrs=127.23.200.254:45999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:23.826813 15212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:23.827037 15212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:23.832137 15254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:23.832159 15256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:23.832147 15252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:23.832417 15212 server_base.cc:1034] running on GCE node
I20260807 20:15:23.832603 15212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:23.832827 15212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:23.833994 15212 hybrid_clock.cc:648] HybridClock initialized: now 1786133723833971 us; error 41 us; skew 500 ppm
Aug 07 20:15:23 dist-test-slave-2j7r krb5kdc[14270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:23.836655 15212 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:15:23.837157 15212 webserver.cc:460] Webserver started at http://127.23.200.193:36785/ using document root <none> and password file <none>
I20260807 20:15:23.837401 15212 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:23.837466 15212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:23.838811 15212 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:15:23.839483 15273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:23.839706 15212 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:15:23.839843 15212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "80664e6c8320444a8cef8c7da0479c34"
format_stamp: "Formatted at 2026-08-07 20:15:23 on dist-test-slave-2j7r"
server_key: "1b618b0016259f48d037fcc56b18101f"
server_key_iv: "d29e91e30019de92f8f2b543fb8fce2f"
server_key_version: "encryptionkey@0"
I20260807 20:15:23.840195 15212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:23.857298 15212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:23.859606 15212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:23.859762 15212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:23.860005 15212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:23.860468 15281 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 20:15:23.862093 15212 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 20:15:23.862142 15212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 20:15:23.862166 15212 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 20:15:23.862807 15212 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 20:15:23.862854 15212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 20:15:23.862919 15281 tablet_bootstrap.cc:492] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34: Bootstrap starting.
I20260807 20:15:23.870433 15212 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:41929
I20260807 20:15:23.870633 15448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:41929 every 8 connection(s)
I20260807 20:15:23.870992 15212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:15:23.874264 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 15212
I20260807 20:15:23.874626 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:38955
--local_ip_for_outbound_sockets=127.23.200.194
--tserver_master_addrs=127.23.200.254:45999
--webserver_port=38813
--webserver_interface=127.23.200.194
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:15:23.874873 15281 log.cc:824] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:23.877272 15281 tablet_bootstrap.cc:492] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:15:23.877532 15281 tablet_bootstrap.cc:492] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34: Bootstrap complete.
I20260807 20:15:23.878327 15281 ts_tablet_manager.cc:1397] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.007s
I20260807 20:15:23.879451 15281 raft_consensus.cc:348] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } }
I20260807 20:15:23.879681 15281 raft_consensus.cc:729] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80664e6c8320444a8cef8c7da0479c34, State: Initialized, Role: FOLLOWER
I20260807 20:15:23.879842 15281 consensus_queue.cc:260] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } }
I20260807 20:15:23.880136 15281 ts_tablet_manager.cc:1428] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
Aug 07 20:15:23 dist-test-slave-2j7r krb5kdc[14270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:23.884420 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.871861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:48229 (local address 127.23.200.254:45999)
0807 20:15:23.873248 (+ 1387us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:23.873261 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:23.873311 (+ 50us) server_negotiation.cc:407] Connection header received
0807 20:15:23.873486 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:23.873495 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:23.873564 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:23.873724 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:23.874673 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.876254 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.877832 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.878353 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.878916 (+ 563us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.880968 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:23.880973 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:23.880974 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:23.880987 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:23.882545 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:23.883051 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:23.883055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:23.883055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:23.883150 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:23.883473 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:23.883474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:23.883474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:23.883723 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:23.883801 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:23.884147 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 20:15:23.884264 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1313,"thread_start_us":93,"threads_started":1}
I20260807 20:15:23.884503 15452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.871739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:48229)
0807 20:15:23.871936 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:23.871985 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:23.872669 (+ 684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:23.873780 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:23.873784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:23.874185 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:23.874457 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.874464 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.876399 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.876401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.877720 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.877723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.878488 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.878489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.878629 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.879222 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:23.879232 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:23.880859 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:23.882638 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:23.882649 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:23.882651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:23.882957 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:23.883212 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:23.883213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:23.883214 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:23.883358 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:23.883798 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:23.883806 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:23.883977 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 20:15:23.884101 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 20:15:23.884287 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":91,"threads_started":1}
I20260807 20:15:23.885301 15450 heartbeater.cc:347] Connected to a master server at 127.23.200.254:45999
I20260807 20:15:23.885438 15450 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:23.885627 15450 heartbeater.cc:511] Master 127.23.200.254:45999 requested a full tablet report, sending...
I20260807 20:15:23.886415 15157 ts_manager.cc:194] Registered new tserver with Master: 80664e6c8320444a8cef8c7da0479c34 (127.23.200.193:41929)
I20260807 20:15:23.887848 15157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:48229
I20260807 20:15:23.888872 15450 heartbeater.cc:503] Master 127.23.200.254:45999 was elected leader, sending a full tablet report...
Aug 07 20:15:23 dist-test-slave-2j7r krb5kdc[14270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for kudu/127.23.200.193@KRBTEST.COM
I20260807 20:15:23.895749 15460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.887218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:41929 (local address 127.0.0.1:57258)
0807 20:15:23.887527 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:23.887568 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:23.887638 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:23.887803 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:23.887808 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:23.888109 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:23.888173 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.888176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.889405 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.889406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.890311 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:23.890314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.890751 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.890751 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:23.890829 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.891275 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:23.891279 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:23.892732 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:23.894573 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:23.894580 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:23.894582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:23.894777 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:23.895031 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:23.895032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:23.895033 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:23.895106 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:23.895431 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:23.895439 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:23.895492 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 20:15:23.895577 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 20:15:23.895677 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":101,"threads_started":1}
I20260807 20:15:23.895807 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:23.887285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57258 (local address 127.23.200.193:41929)
0807 20:15:23.887468 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:23.887471 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:23.887638 (+ 167us) server_negotiation.cc:407] Connection header received
0807 20:15:23.887722 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:23.887724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:23.887739 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:23.887796 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:23.888240 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.889242 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.890394 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:23.890678 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:23.890734 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:23.892809 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:23.892818 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:23.892818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:23.892825 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:23.894490 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:23.894846 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:23.894849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:23.894850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:23.894912 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:23.895168 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:23.895169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:23.895170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:23.895346 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:23.895424 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:23.895615 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 20:15:23.895713 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":70,"threads_started":1}
I20260807 20:15:23.896749 15342 tablet_service.cc:1514] Processing DeleteTablet for tablet 2429853b38b742f4a396964f26893df4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:15:23 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:57258
I20260807 20:15:23.897409 15466 tablet_replica.cc:330] stopping tablet replica
I20260807 20:15:23.897517 15466 raft_consensus.cc:2229] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:15:23.897586 15466 raft_consensus.cc:2258] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:15:23.897907 15466 ts_tablet_manager.cc:1905] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:15:23.900157 15466 ts_tablet_manager.cc:1918] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:15:23.900244 15466 log.cc:1196] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal/wals/2429853b38b742f4a396964f26893df4
I20260807 20:15:23.900460 15466 ts_tablet_manager.cc:1939] T 2429853b38b742f4a396964f26893df4 P 80664e6c8320444a8cef8c7da0479c34: Deleting consensus metadata
I20260807 20:15:23.900967 15134 catalog_manager.cc:4841] TS 80664e6c8320444a8cef8c7da0479c34 (127.23.200.193:41929): tablet 2429853b38b742f4a396964f26893df4 (table test-table [id=8e4c7eb5b97a4060a21006a633174d39]) successfully deleted
W20260807 20:15:23.967931 15455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:23.968142 15455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:23.968164 15455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:23.969360 15455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:23.969403 15455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:23.969427 15455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:15:23.970875 15455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:38955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=38813
--tserver_master_addrs=127.23.200.254:45999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:23.971074 15455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:23.971264 15455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:23.976401 15481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:23.976411 15478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:23.976377 15479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:23.976447 15455 server_base.cc:1034] running on GCE node
I20260807 20:15:23.976806 15455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:23.977046 15455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:23.978247 15455 hybrid_clock.cc:648] HybridClock initialized: now 1786133723978204 us; error 53 us; skew 500 ppm
Aug 07 20:15:23 dist-test-slave-2j7r krb5kdc[14270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:23.980962 15455 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:15:23.981474 15455 webserver.cc:460] Webserver started at http://127.23.200.194:38813/ using document root <none> and password file <none>
I20260807 20:15:23.981703 15455 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:23.981801 15455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:23.983145 15455 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:15:23.983743 15495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:23.983944 15455 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:15:23.984030 15455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "3e3cfea450f6430ca1eb4000d703af9f"
format_stamp: "Formatted at 2026-08-07 20:15:23 on dist-test-slave-2j7r"
server_key: "d82e5afb4dd2b070cf5ff6c76f250a54"
server_key_iv: "ccc14b27761254297975ead5b8ea9e6c"
server_key_version: "encryptionkey@0"
I20260807 20:15:23.984368 15455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:23.999702 15455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:24.001507 15455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:24.001639 15455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:24.001850 15455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:24.002142 15455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:15:24.002182 15455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:24.002213 15455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:24.002238 15455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:24.007768 15455 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:38955
I20260807 20:15:24.007829 15673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:38955 every 8 connection(s)
I20260807 20:15:24.008158 15455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
Aug 07 20:15:24 dist-test-slave-2j7r krb5kdc[14270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:24.014973 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 15455
I20260807 20:15:24.015269 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:41701
--local_ip_for_outbound_sockets=127.23.200.195
--tserver_master_addrs=127.23.200.254:45999
--webserver_port=39379
--webserver_interface=127.23.200.195
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:15:24.017304 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.008930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:44961 (local address 127.23.200.254:45999)
0807 20:15:24.009001 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:24.009003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:24.009154 (+ 151us) server_negotiation.cc:407] Connection header received
0807 20:15:24.009644 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:24.009646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:24.009661 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:24.009719 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:24.010670 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.011182 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.012173 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.012384 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.012443 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:24.014532 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:24.014534 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:24.014534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:24.014542 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:24.015961 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:24.016332 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:24.016334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:24.016334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:24.016424 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:24.016647 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:24.016648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:24.016648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:24.016756 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:24.016811 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:24.017078 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 20:15:24.017202 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:15:24.017498 15679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.008853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:44961)
0807 20:15:24.009009 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:24.009052 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:24.009537 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:24.009780 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:24.009783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:24.010360 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:24.010556 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:24.010561 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.011267 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.011268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:24.012086 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:24.012089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.012448 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.012448 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:24.012553 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:24.012985 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:24.012994 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:24.014437 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:24.016040 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:24.016047 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:24.016049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:24.016265 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:24.016486 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:24.016487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:24.016488 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:24.016592 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:24.016813 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:24.016819 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:24.016959 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 20:15:24.017048 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 20:15:24.017224 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
I20260807 20:15:24.017994 15674 heartbeater.cc:347] Connected to a master server at 127.23.200.254:45999
I20260807 20:15:24.018103 15674 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:24.018299 15674 heartbeater.cc:511] Master 127.23.200.254:45999 requested a full tablet report, sending...
I20260807 20:15:24.018697 15157 ts_manager.cc:194] Registered new tserver with Master: 3e3cfea450f6430ca1eb4000d703af9f (127.23.200.194:38955)
I20260807 20:15:24.019063 15157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:44961
W20260807 20:15:24.100484 15681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:24.100711 15681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:24.100747 15681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:24.101969 15681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:24.102017 15681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:24.102056 15681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:15:24.103531 15681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:41701
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=39379
--tserver_master_addrs=127.23.200.254:45999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:24.103788 15681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:24.104007 15681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:24.109012 15687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:24.109005 15692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:24.109004 15689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:24.109329 15681 server_base.cc:1034] running on GCE node
I20260807 20:15:24.109512 15681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:24.109699 15681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:24.110853 15681 hybrid_clock.cc:648] HybridClock initialized: now 1786133724110839 us; error 29 us; skew 500 ppm
Aug 07 20:15:24 dist-test-slave-2j7r krb5kdc[14270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133724, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:24.113396 15681 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:15:24.113821 15681 webserver.cc:460] Webserver started at http://127.23.200.195:39379/ using document root <none> and password file <none>
I20260807 20:15:24.114014 15681 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:24.114069 15681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:24.115407 15681 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:15:24.115948 15702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:24.116096 15681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:15:24.116181 15681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "9b03878eb76d4c4d9ad14bb3dd221eea"
format_stamp: "Formatted at 2026-08-07 20:15:23 on dist-test-slave-2j7r"
server_key: "a386e610d1087ba33857f2b2bb86c7be"
server_key_iv: "906d76d890f2dee9820477d491e24dbd"
server_key_version: "encryptionkey@0"
I20260807 20:15:24.116520 15681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:24.127147 15681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:24.129245 15681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:24.129369 15681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:24.129559 15681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:24.129818 15681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:15:24.129848 15681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:24.129868 15681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:24.129879 15681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:24.135219 15681 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:41701
I20260807 20:15:24.135303 15883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:41701 every 8 connection(s)
I20260807 20:15:24.135586 15681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
Aug 07 20:15:24 dist-test-slave-2j7r krb5kdc[14270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133724, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:24.141703 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 15681
I20260807 20:15:24.144912 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.136284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:49049 (local address 127.23.200.254:45999)
0807 20:15:24.136348 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:24.136350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:24.136539 (+ 189us) server_negotiation.cc:407] Connection header received
0807 20:15:24.137063 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:24.137064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:24.137078 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:24.137149 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:24.138099 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.138599 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.139559 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.139762 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.139818 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:24.141992 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:24.141994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:24.141994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:24.142002 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:24.143514 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:24.143953 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:24.143955 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:24.143955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:24.144020 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:24.144287 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:24.144288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:24.144289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:24.144429 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:24.144493 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:24.144727 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 20:15:24.144814 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 20:15:24.145030 15889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.136262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:49049)
0807 20:15:24.136421 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:24.136457 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:24.136960 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:24.137236 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:24.137241 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:24.137799 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:24.137995 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:24.138000 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.138680 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.138682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:24.139470 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:24.139473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.139831 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.139832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:24.139956 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:24.140461 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:24.140469 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:24.141887 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:24.143642 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:24.143650 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:24.143652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:24.143885 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:24.144085 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:24.144086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:24.144086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:24.144222 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:24.144495 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:24.144499 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:24.144613 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 20:15:24.144690 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 20:15:24.144809 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260807 20:15:24.145539 15886 heartbeater.cc:347] Connected to a master server at 127.23.200.254:45999
I20260807 20:15:24.145632 15886 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:24.145813 15886 heartbeater.cc:511] Master 127.23.200.254:45999 requested a full tablet report, sending...
I20260807 20:15:24.146232 15157 ts_manager.cc:194] Registered new tserver with Master: 9b03878eb76d4c4d9ad14bb3dd221eea (127.23.200.195:41701)
I20260807 20:15:24.146584 15157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:49049
I20260807 20:15:24.148706 15899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.142064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34810 (local address 127.23.200.254:45999)
0807 20:15:24.142184 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:24.142186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:24.142212 (+ 26us) server_negotiation.cc:407] Connection header received
0807 20:15:24.142283 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:24.142284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:24.142295 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:24.142369 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:24.143063 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.143594 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.144852 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.145195 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.145267 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:24.146477 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:24.146478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:24.146478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:24.146485 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:24.147854 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:24.148156 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:24.148156 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:24.148157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:24.148201 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:24.148337 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:24.148337 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:24.148338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:24.148426 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:24.148540 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:24.148567 (+ 27us) server_negotiation.cc:299] Negotiation successful
0807 20:15:24.148624 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260807 20:15:24.149406 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:15:24.152015 15899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.149703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34816 (local address 127.23.200.254:45999)
0807 20:15:24.149754 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:24.149756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:24.149806 (+ 50us) server_negotiation.cc:407] Connection header received
0807 20:15:24.149877 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:24.149878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:24.149913 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:24.149972 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:15:24.150092 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.150535 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.151310 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.151522 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.151573 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:24.151736 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:15:24.151827 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:15:24.151891 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:24.151934 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 20:15:24.151955 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:15:24.152251 15157 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:34816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:15:24.153869 15157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:15:24.156184 15342 tablet_service.cc:1467] Processing CreateTablet for tablet c579b1d58b414c9f9b3f1e263dca7797 (DEFAULT_TABLE table=test-table [id=e8ad90bb5e754386a136d2ebbd8ff780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:15:24.156409 15342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c579b1d58b414c9f9b3f1e263dca7797. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:24.157852 15281 tablet_bootstrap.cc:492] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34: Bootstrap starting.
I20260807 20:15:24.158196 15281 tablet_bootstrap.cc:654] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:24.159198 15281 tablet_bootstrap.cc:492] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34: No bootstrap required, opened a new log
I20260807 20:15:24.159252 15281 ts_tablet_manager.cc:1397] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:15:24.159390 15281 raft_consensus.cc:348] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } }
I20260807 20:15:24.159461 15281 raft_consensus.cc:374] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:24.159502 15281 raft_consensus.cc:729] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80664e6c8320444a8cef8c7da0479c34, State: Initialized, Role: FOLLOWER
I20260807 20:15:24.159546 15281 consensus_queue.cc:260] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } }
I20260807 20:15:24.159633 15281 ts_tablet_manager.cc:1428] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:24.159642 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.155682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:41701 (local address 127.0.0.1:47502)
0807 20:15:24.155960 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:24.155970 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:24.156025 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:24.156254 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:24.156255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:24.156257 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:24.156298 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:24.156300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.157537 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.157539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:24.158590 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:24.158593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.159394 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.159395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:24.159456 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:24.159475 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 20:15:24.159526 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 20:15:24.159566 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":25,"thread_start_us":118,"threads_started":1}
I20260807 20:15:24.159652 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.155890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47502 (local address 127.23.200.195:41701)
0807 20:15:24.156061 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:24.156064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:24.156073 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:15:24.156105 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:24.156107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:24.156120 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:24.156184 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:24.156368 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.157427 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.158668 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.159279 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.159381 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:24.159441 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:24.159546 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 20:15:24.159580 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260807 20:15:24.159801 15460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.155659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:38955 (local address 127.0.0.1:47510)
0807 20:15:24.155759 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:24.155769 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:24.155806 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:24.156146 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:24.156147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:24.156149 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:24.156189 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:24.156191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.157576 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.157577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:24.158676 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:24.158680 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.159590 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.159591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:24.159662 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:24.159666 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:15:24.159698 (+ 32us) client_negotiation.cc:240] Negotiation successful
0807 20:15:24.159740 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":41}
I20260807 20:15:24.160108 15903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.155779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47510 (local address 127.23.200.194:38955)
0807 20:15:24.155949 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:24.155953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:24.155960 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:15:24.155996 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:24.155998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:24.156015 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:24.156125 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:24.156254 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.157497 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.158779 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.159489 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.159900 (+ 411us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:24.159969 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:24.159994 (+ 25us) server_negotiation.cc:299] Negotiation successful
0807 20:15:24.160033 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260807 20:15:24.160475 15782 tablet_service.cc:1467] Processing CreateTablet for tablet c579b1d58b414c9f9b3f1e263dca7797 (DEFAULT_TABLE table=test-table [id=e8ad90bb5e754386a136d2ebbd8ff780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:15:24.160816 15782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c579b1d58b414c9f9b3f1e263dca7797. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:24.160816 15569 tablet_service.cc:1467] Processing CreateTablet for tablet c579b1d58b414c9f9b3f1e263dca7797 (DEFAULT_TABLE table=test-table [id=e8ad90bb5e754386a136d2ebbd8ff780]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:15:24.161152 15569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c579b1d58b414c9f9b3f1e263dca7797. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:24.164345 15907 tablet_bootstrap.cc:492] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea: Bootstrap starting.
I20260807 20:15:24.165005 15907 tablet_bootstrap.cc:654] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:24.165485 15907 log.cc:824] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:24.165788 15908 tablet_bootstrap.cc:492] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f: Bootstrap starting.
I20260807 20:15:24.166321 15908 tablet_bootstrap.cc:654] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:24.166780 15908 log.cc:824] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:24.167259 15907 tablet_bootstrap.cc:492] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea: No bootstrap required, opened a new log
I20260807 20:15:24.167330 15907 ts_tablet_manager.cc:1397] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 20:15:24.167634 15908 tablet_bootstrap.cc:492] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f: No bootstrap required, opened a new log
I20260807 20:15:24.167699 15908 ts_tablet_manager.cc:1397] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:15:24.168828 15907 raft_consensus.cc:348] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } }
I20260807 20:15:24.168969 15907 raft_consensus.cc:374] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:24.169001 15907 raft_consensus.cc:729] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b03878eb76d4c4d9ad14bb3dd221eea, State: Initialized, Role: FOLLOWER
I20260807 20:15:24.169024 15908 raft_consensus.cc:348] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } }
I20260807 20:15:24.169140 15907 consensus_queue.cc:260] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } }
I20260807 20:15:24.169214 15908 raft_consensus.cc:374] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:24.169255 15908 raft_consensus.cc:729] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3cfea450f6430ca1eb4000d703af9f, State: Initialized, Role: FOLLOWER
I20260807 20:15:24.169368 15908 consensus_queue.cc:260] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } }
I20260807 20:15:24.169432 15907 ts_tablet_manager.cc:1428] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:15:24.169453 15886 heartbeater.cc:503] Master 127.23.200.254:45999 was elected leader, sending a full tablet report...
I20260807 20:15:24.169646 15908 ts_tablet_manager.cc:1428] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:15:24.169668 15674 heartbeater.cc:503] Master 127.23.200.254:45999 was elected leader, sending a full tablet report...
I20260807 20:15:24.189977 15458 raft_consensus.cc:482] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:15:24.190119 15458 raft_consensus.cc:504] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } }
I20260807 20:15:24.190476 15458 leader_election.cc:288] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e3cfea450f6430ca1eb4000d703af9f (127.23.200.194:38955), 9b03878eb76d4c4d9ad14bb3dd221eea (127.23.200.195:41701)
I20260807 20:15:24.194291 15903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.190704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:42405 (local address 127.23.200.194:38955)
0807 20:15:24.190752 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:24.190754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:24.190929 (+ 175us) server_negotiation.cc:407] Connection header received
0807 20:15:24.190960 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:24.190961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:24.190971 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:24.191022 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:24.191142 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.191855 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.193312 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.193878 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.193954 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:24.193962 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:24.194183 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 20:15:24.194219 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:15:24.194290 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.190716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:38607 (local address 127.23.200.195:41701)
0807 20:15:24.190763 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:24.190764 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:24.190773 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:15:24.190872 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:24.190873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:24.190901 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:24.190961 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:24.191117 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.191856 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.193311 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.193871 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.193953 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:24.193962 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:24.194184 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 20:15:24.194220 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 20:15:24.194294 15452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.190628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:41701 (local address 127.23.200.193:38607)
0807 20:15:24.190679 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:24.190717 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:24.190779 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:24.190972 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:24.190972 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:24.190974 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:24.191049 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:24.191052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.191931 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.191933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:24.193235 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:24.193239 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.193978 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.193979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:24.194071 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:24.194101 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 20:15:24.194163 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 20:15:24.194209 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":60}
I20260807 20:15:24.194394 15916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.190628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:38955 (local address 127.23.200.193:42405)
0807 20:15:24.190863 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:24.190872 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:24.190920 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:24.191030 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:24.191030 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:24.191032 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:24.191083 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:24.191085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.191931 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.191933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:24.193235 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:24.193239 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.194004 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.194005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:24.194071 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:24.194101 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 20:15:24.194163 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 20:15:24.194209 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":96,"thread_start_us":127,"threads_started":1}
I20260807 20:15:24.194710 15591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c579b1d58b414c9f9b3f1e263dca7797" candidate_uuid: "80664e6c8320444a8cef8c7da0479c34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3cfea450f6430ca1eb4000d703af9f" is_pre_election: true
I20260807 20:15:24.194896 15591 raft_consensus.cc:2454] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80664e6c8320444a8cef8c7da0479c34 in term 0.
I20260807 20:15:24.194854 15809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c579b1d58b414c9f9b3f1e263dca7797" candidate_uuid: "80664e6c8320444a8cef8c7da0479c34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" is_pre_election: true
I20260807 20:15:24.195016 15809 raft_consensus.cc:2454] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80664e6c8320444a8cef8c7da0479c34 in term 0.
I20260807 20:15:24.195142 15276 leader_election.cc:302] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3cfea450f6430ca1eb4000d703af9f, 80664e6c8320444a8cef8c7da0479c34; no voters:
I20260807 20:15:24.195286 15458 raft_consensus.cc:2781] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:15:24.195346 15458 raft_consensus.cc:482] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:15:24.195366 15458 raft_consensus.cc:3037] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:24.196058 15458 raft_consensus.cc:504] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } }
I20260807 20:15:24.196177 15458 leader_election.cc:288] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [CANDIDATE]: Term 1 election: Requested vote from peers 3e3cfea450f6430ca1eb4000d703af9f (127.23.200.194:38955), 9b03878eb76d4c4d9ad14bb3dd221eea (127.23.200.195:41701)
I20260807 20:15:24.196338 15591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c579b1d58b414c9f9b3f1e263dca7797" candidate_uuid: "80664e6c8320444a8cef8c7da0479c34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e3cfea450f6430ca1eb4000d703af9f"
I20260807 20:15:24.196419 15591 raft_consensus.cc:3037] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:24.196480 15809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c579b1d58b414c9f9b3f1e263dca7797" candidate_uuid: "80664e6c8320444a8cef8c7da0479c34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea"
I20260807 20:15:24.196570 15809 raft_consensus.cc:3037] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:24.197257 15591 raft_consensus.cc:2454] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80664e6c8320444a8cef8c7da0479c34 in term 1.
I20260807 20:15:24.197412 15276 leader_election.cc:302] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e3cfea450f6430ca1eb4000d703af9f, 80664e6c8320444a8cef8c7da0479c34; no voters:
I20260807 20:15:24.197424 15809 raft_consensus.cc:2454] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80664e6c8320444a8cef8c7da0479c34 in term 1.
I20260807 20:15:24.197513 15458 raft_consensus.cc:2781] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:15:24.197678 15458 raft_consensus.cc:686] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [term 1 LEADER]: Becoming Leader. State: Replica: 80664e6c8320444a8cef8c7da0479c34, State: Running, Role: LEADER
I20260807 20:15:24.197765 15458 consensus_queue.cc:237] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } }
I20260807 20:15:24.198532 15157 catalog_manager.cc:5495] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80664e6c8320444a8cef8c7da0479c34 (127.23.200.193). New cstate: current_term: 1 leader_uuid: "80664e6c8320444a8cef8c7da0479c34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80664e6c8320444a8cef8c7da0479c34" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 41929 } health_report { overall_health: HEALTHY } } }
I20260807 20:15:24.210007 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.207446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57262 (local address 127.23.200.193:41929)
0807 20:15:24.207517 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:24.207519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:24.207532 (+ 13us) server_negotiation.cc:407] Connection header received
0807 20:15:24.207580 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:24.207581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:24.207607 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:24.207677 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:15:24.207842 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.208375 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.209192 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:24.209416 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:24.209481 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:24.209649 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:15:24.209764 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:15:24.209833 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:24.209887 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 20:15:24.209914 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:15:24.211707 15591 raft_consensus.cc:1261] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f [term 1 FOLLOWER]: Refusing update from remote peer 80664e6c8320444a8cef8c7da0479c34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:15:24.211724 15809 raft_consensus.cc:1261] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 1 FOLLOWER]: Refusing update from remote peer 80664e6c8320444a8cef8c7da0479c34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:15:24.212083 15917 consensus_queue.cc:1035] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e3cfea450f6430ca1eb4000d703af9f" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:15:24.212248 15458 consensus_queue.cc:1035] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b03878eb76d4c4d9ad14bb3dd221eea" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 41701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:15:24.214679 15920 mvcc.cc:205] Tried to move back new op lower bound from 7316003734368436224 to 7316003734314577920. Current Snapshot: MvccSnapshot[applied={T|T < 7316003734368436224}]
I20260807 20:15:24.215237 15921 mvcc.cc:205] Tried to move back new op lower bound from 7316003734368436224 to 7316003734314577920. Current Snapshot: MvccSnapshot[applied={T|T < 7316003734368436224 or (T in {7316003734368436224})}]
I20260807 20:15:24.217056 15155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:34816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:15:24.217175 15155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:34816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:15:24.217864 15155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff: Sending DeleteTablet for 3 replicas of tablet c579b1d58b414c9f9b3f1e263dca7797
I20260807 20:15:24.218178 15342 tablet_service.cc:1514] Processing DeleteTablet for tablet c579b1d58b414c9f9b3f1e263dca7797 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:15:24 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:57258
I20260807 20:15:24.218218 15569 tablet_service.cc:1514] Processing DeleteTablet for tablet c579b1d58b414c9f9b3f1e263dca7797 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:15:24 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:47510
I20260807 20:15:24.218271 15782 tablet_service.cc:1514] Processing DeleteTablet for tablet c579b1d58b414c9f9b3f1e263dca7797 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:15:24 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:47502
I20260807 20:15:24.218438 15466 tablet_replica.cc:330] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34: stopping tablet replica
I20260807 20:15:24.218461 15934 tablet_replica.cc:330] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea: stopping tablet replica
I20260807 20:15:24.218498 15466 raft_consensus.cc:2229] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:15:24.218489 15933 tablet_replica.cc:330] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f: stopping tablet replica
I20260807 20:15:24.218542 15934 raft_consensus.cc:2229] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:15:24.218580 15466 raft_consensus.cc:2258] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:15:24.218595 15933 raft_consensus.cc:2229] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:15:24.218640 15934 raft_consensus.cc:2258] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:15:24.218690 15933 raft_consensus.cc:2258] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:15:24.219094 15466 ts_tablet_manager.cc:1905] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:15:24.219199 15933 ts_tablet_manager.cc:1905] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:15:24.219193 15934 ts_tablet_manager.cc:1905] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:15:24.220780 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 15212
I20260807 20:15:24.220817 15934 ts_tablet_manager.cc:1918] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:15:24.220875 15934 log.cc:1196] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/c579b1d58b414c9f9b3f1e263dca7797
I20260807 20:15:24.220909 15933 ts_tablet_manager.cc:1918] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:15:24.220978 15933 log.cc:1196] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/c579b1d58b414c9f9b3f1e263dca7797
I20260807 20:15:24.221156 15934 ts_tablet_manager.cc:1939] T c579b1d58b414c9f9b3f1e263dca7797 P 9b03878eb76d4c4d9ad14bb3dd221eea: Deleting consensus metadata
I20260807 20:15:24.221360 15933 ts_tablet_manager.cc:1939] T c579b1d58b414c9f9b3f1e263dca7797 P 3e3cfea450f6430ca1eb4000d703af9f: Deleting consensus metadata
I20260807 20:15:24.221684 15135 catalog_manager.cc:4841] TS 9b03878eb76d4c4d9ad14bb3dd221eea (127.23.200.195:41701): tablet c579b1d58b414c9f9b3f1e263dca7797 (table test-table [id=e8ad90bb5e754386a136d2ebbd8ff780]) successfully deleted
I20260807 20:15:24.221807 15134 catalog_manager.cc:4841] TS 3e3cfea450f6430ca1eb4000d703af9f (127.23.200.194:38955): tablet c579b1d58b414c9f9b3f1e263dca7797 (table test-table [id=e8ad90bb5e754386a136d2ebbd8ff780]) successfully deleted
W20260807 20:15:24.234688 15134 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:41929 (error 108)
I20260807 20:15:24.234880 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 15455
I20260807 20:15:24.241511 15460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.241324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:41929 (local address 127.0.0.1:57270)
0807 20:15:24.241404 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:24.241470 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:41929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 20:15:24.241693 15134 catalog_manager.cc:4584] TS 80664e6c8320444a8cef8c7da0479c34 (127.23.200.193:41929): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c579b1d58b414c9f9b3f1e263dca7797: Network error: Client connection negotiation failed: client connection to 127.23.200.193:41929: connect: Connection refused (error 111)
I20260807 20:15:24.245232 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 15681
I20260807 20:15:24.259583 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 15105
I20260807 20:15:24.271289 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:45999
--webserver_interface=127.23.200.254
--webserver_port=36971
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:45999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:24.353684 15938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:24.353891 15938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:24.353914 15938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:24.355046 15938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:15:24.355088 15938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:24.355101 15938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:24.355113 15938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:15:24.355123 15938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:15:24.356604 15938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:45999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:45999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=36971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:24.356839 15938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:24.357067 15938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:15:24.362083 15938 server_base.cc:1034] running on GCE node
W20260807 20:15:24.362075 15949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:24.362061 15947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:24.362063 15952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:24.362454 15938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:24.362702 15938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:24.363899 15938 hybrid_clock.cc:648] HybridClock initialized: now 1786133724363879 us; error 35 us; skew 500 ppm
Aug 07 20:15:24 dist-test-slave-2j7r krb5kdc[14270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133724, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:24.366441 15938 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:15:24.366906 15938 webserver.cc:460] Webserver started at http://127.23.200.254:36971/ using document root <none> and password file <none>
I20260807 20:15:24.367096 15938 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:24.367138 15938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:24.368623 15938 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:15:24.369429 15960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:24.369622 15938 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:15:24.369714 15938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "2253a25b07cb4d8b876b6b8c2571eaff"
format_stamp: "Formatted at 2026-08-07 20:15:22 on dist-test-slave-2j7r"
server_key: "63343f36d69f242819cfb44bdc296c31"
server_key_iv: "03c08f933ccad33029c51f84342496a2"
server_key_version: "encryptionkey@0"
I20260807 20:15:24.370066 15938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:24.382182 15938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:24.384262 15938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:24.384398 15938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:24.388424 15938 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:45999
I20260807 20:15:24.388491 16039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:45999 every 8 connection(s)
I20260807 20:15:24.388854 15938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:15:24.390486 16040 sys_catalog.cc:263] Verifying existing consensus state
I20260807 20:15:24.391202 16040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff: Bootstrap starting.
I20260807 20:15:24.396945 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 15938
I20260807 20:15:24.397337 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:41929
--local_ip_for_outbound_sockets=127.23.200.193
--tserver_master_addrs=127.23.200.254:45999
--webserver_port=36785
--webserver_interface=127.23.200.193
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:15:24.399827 16040 log.cc:824] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:24.402207 16040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:15:24.402479 16040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff: Bootstrap complete.
I20260807 20:15:24.404661 16040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } }
I20260807 20:15:24.404901 16040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2253a25b07cb4d8b876b6b8c2571eaff, State: Initialized, Role: FOLLOWER
I20260807 20:15:24.405073 16040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } }
I20260807 20:15:24.405174 16040 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:15:24.405226 16040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:15:24.405275 16040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 2 FOLLOWER]: Advancing to term 3
I20260807 20:15:24.406297 16040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } }
I20260807 20:15:24.406404 16040 leader_election.cc:302] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2253a25b07cb4d8b876b6b8c2571eaff; no voters:
I20260807 20:15:24.406589 16040 leader_election.cc:288] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 20:15:24.406643 16046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 3 FOLLOWER]: Leader election won for term 3
I20260807 20:15:24.406761 16046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [term 3 LEADER]: Becoming Leader. State: Replica: 2253a25b07cb4d8b876b6b8c2571eaff, State: Running, Role: LEADER
I20260807 20:15:24.406868 16046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } }
I20260807 20:15:24.406940 16040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:15:24.407166 16046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2253a25b07cb4d8b876b6b8c2571eaff. Latest consensus state: current_term: 3 leader_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } } }
I20260807 20:15:24.407265 16046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:24.407436 16047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2253a25b07cb4d8b876b6b8c2571eaff" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45999 } } }
I20260807 20:15:24.407509 16047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2253a25b07cb4d8b876b6b8c2571eaff [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:24.407811 16051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:15:24.408648 16051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:15:24.408753 16051 catalog_manager.cc:1259] Loaded cluster ID: ce5e9117a1934553a4f7d37a6c4daaee
I20260807 20:15:24.408787 16051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 20:15:24.409135 16051 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fdf9dff89ad google::LogMessage::Fail() at ??:0
@ 0x7fdf9dffb88a google::LogMessage::SendToLog() at ??:0
@ 0x7fdf9dff84de google::LogMessage::Flush() at ??:0
@ 0x7fdf9dff9a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fdfa5f1d0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fdfa5f268b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fdf9fec60d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fdf9febb954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdfa0d1f6db start_thread at ??:0
@ 0x7fdf9caf661f clone at ??:0
W20260807 20:15:24.495420 16044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:24.495627 16044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:24.495651 16044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:24.496801 16044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:24.496842 16044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:24.496869 16044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:15:24.498445 16044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:41929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=36785
--tserver_master_addrs=127.23.200.254:45999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:24.498687 16044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:24.498927 16044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:24.503937 16110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:24.503986 16044 server_base.cc:1034] running on GCE node
W20260807 20:15:24.503952 16104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:24.504117 16106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:24.504403 16044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:24.504650 16044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:24.505815 16044 hybrid_clock.cc:648] HybridClock initialized: now 1786133724505800 us; error 33 us; skew 500 ppm
Aug 07 20:15:24 dist-test-slave-2j7r krb5kdc[14270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133724, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:24.508411 16044 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:15:24.508872 16044 webserver.cc:460] Webserver started at http://127.23.200.193:36785/ using document root <none> and password file <none>
I20260807 20:15:24.509128 16044 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:24.509186 16044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:24.510535 16044 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:15:24.511193 16120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:24.511351 16044 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:15:24.511441 16044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "80664e6c8320444a8cef8c7da0479c34"
format_stamp: "Formatted at 2026-08-07 20:15:23 on dist-test-slave-2j7r"
server_key: "1b618b0016259f48d037fcc56b18101f"
server_key_iv: "d29e91e30019de92f8f2b543fb8fce2f"
server_key_version: "encryptionkey@0"
I20260807 20:15:24.511780 16044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:24.527509 16044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:24.529675 16044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:24.529798 16044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:24.530028 16044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:24.530480 16130 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260807 20:15:24.532462 16130 ts_tablet_manager.cc:1852] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260807 20:15:24.532866 16130 ts_tablet_manager.cc:1905] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:15:24.535555 16130 ts_tablet_manager.cc:1918] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:15:24.535653 16130 log.cc:1196] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/wal/wals/c579b1d58b414c9f9b3f1e263dca7797
I20260807 20:15:24.535953 16130 ts_tablet_manager.cc:1939] T c579b1d58b414c9f9b3f1e263dca7797 P 80664e6c8320444a8cef8c7da0479c34: Deleting consensus metadata
I20260807 20:15:24.536173 16044 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260807 20:15:24.536221 16044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260807 20:15:24.536254 16044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:24.536279 16044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:24.543180 16044 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:41929
I20260807 20:15:24.543257 16309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:41929 every 8 connection(s)
I20260807 20:15:24.543558 16044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:15:24.543836 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 16044
I20260807 20:15:24.544147 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:38955
--local_ip_for_outbound_sockets=127.23.200.194
--tserver_master_addrs=127.23.200.254:45999
--webserver_port=38813
--webserver_interface=127.23.200.194
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:24.607070 16314 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:15:24.544299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:55787)
0807 20:15:24.545242 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:24.545288 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:24.545858 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:24.606853 (+ 60995us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":891,"thread_start_us":85,"threads_started":1}
W20260807 20:15:24.607362 16310 heartbeater.cc:644] Failed to heartbeat to 127.23.200.254:45999 (0 consecutive failures): Network error: Failed to ping master at 127.23.200.254:45999: Client connection negotiation failed: client connection to 127.23.200.254:45999: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 20:15:24.607754 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.607642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:44539)
0807 20:15:24.607693 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:24.607731 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 20:15:24.608026 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.607954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:39263)
0807 20:15:24.607988 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:24.608010 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 20:15:24.608120 16310 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 20:15:24.644234 16315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:24.644447 16315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:24.644483 16315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:24.645752 16315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:24.645807 16315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:24.645846 16315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:15:24.647389 16315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:38955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=38813
--tserver_master_addrs=127.23.200.254:45999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:24.647656 16315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:24.647899 16315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:24.652880 16322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:24.652922 16324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:24.652987 16315 server_base.cc:1034] running on GCE node
W20260807 20:15:24.653071 16327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:24.653319 16315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:24.653574 16315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:24.654708 16315 hybrid_clock.cc:648] HybridClock initialized: now 1786133724654697 us; error 27 us; skew 500 ppm
Aug 07 20:15:24 dist-test-slave-2j7r krb5kdc[14270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133724, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:24.657158 16315 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:15:24.657600 16315 webserver.cc:460] Webserver started at http://127.23.200.194:38813/ using document root <none> and password file <none>
I20260807 20:15:24.657814 16315 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:24.657863 16315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:24.659140 16315 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:15:24.659703 16338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:24.659857 16315 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:15:24.659950 16315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "3e3cfea450f6430ca1eb4000d703af9f"
format_stamp: "Formatted at 2026-08-07 20:15:23 on dist-test-slave-2j7r"
server_key: "d82e5afb4dd2b070cf5ff6c76f250a54"
server_key_iv: "ccc14b27761254297975ead5b8ea9e6c"
server_key_version: "encryptionkey@0"
I20260807 20:15:24.660274 16315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:24.671413 16315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:24.673537 16315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:24.673676 16315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:24.673882 16315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:24.674180 16315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:15:24.674221 16315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:24.674249 16315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:24.674273 16315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:24.679884 16315 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:38955
I20260807 20:15:24.679943 16514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:38955 every 8 connection(s)
I20260807 20:15:24.680261 16315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:15:24.681378 16519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.680931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:44463)
0807 20:15:24.681076 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:24.681181 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
W20260807 20:15:24.681742 16516 heartbeater.cc:644] Failed to heartbeat to 127.23.200.254:45999 (0 consecutive failures): Network error: Failed to ping master at 127.23.200.254:45999: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
I20260807 20:15:24.681988 16519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.681908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:56009)
0807 20:15:24.681946 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:24.681971 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260807 20:15:24.682191 16519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.682135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:33253)
0807 20:15:24.682163 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:24.682180 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260807 20:15:24.682288 16516 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 20:15:24.683699 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 16315
I20260807 20:15:24.683931 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:41701
--local_ip_for_outbound_sockets=127.23.200.195
--tserver_master_addrs=127.23.200.254:45999
--webserver_port=39379
--webserver_interface=127.23.200.195
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:24.767652 16520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:24.767851 16520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:24.767874 16520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:24.768946 16520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:24.768978 16520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:24.769003 16520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:15:24.770502 16520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:41701
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=39379
--tserver_master_addrs=127.23.200.254:45999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:24.770685 16520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:24.770861 16520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:15:24.775887 16520 server_base.cc:1034] running on GCE node
W20260807 20:15:24.775857 16526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:24.775857 16527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:24.775851 16530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:24.776217 16520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:24.776467 16520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:24.777624 16520 hybrid_clock.cc:648] HybridClock initialized: now 1786133724777611 us; error 28 us; skew 500 ppm
Aug 07 20:15:24 dist-test-slave-2j7r krb5kdc[14270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133724, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:24.780189 16520 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:15:24.780629 16520 webserver.cc:460] Webserver started at http://127.23.200.195:39379/ using document root <none> and password file <none>
I20260807 20:15:24.780853 16520 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:24.780901 16520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:24.782354 16520 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:15:24.783012 16545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:24.783155 16520 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:15:24.783237 16520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "9b03878eb76d4c4d9ad14bb3dd221eea"
format_stamp: "Formatted at 2026-08-07 20:15:23 on dist-test-slave-2j7r"
server_key: "a386e610d1087ba33857f2b2bb86c7be"
server_key_iv: "906d76d890f2dee9820477d491e24dbd"
server_key_version: "encryptionkey@0"
I20260807 20:15:24.783617 16520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:24.793795 16520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:24.795863 16520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:24.796001 16520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:24.796221 16520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:24.796520 16520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:15:24.796554 16520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:24.796582 16520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:24.796603 16520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:24.802206 16520 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:41701
I20260807 20:15:24.802284 16710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:41701 every 8 connection(s)
I20260807 20:15:24.802599 16520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:15:24.803684 16713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.803277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:41955)
0807 20:15:24.803446 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:24.803528 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
W20260807 20:15:24.804080 16711 heartbeater.cc:644] Failed to heartbeat to 127.23.200.254:45999 (0 consecutive failures): Network error: Failed to ping master at 127.23.200.254:45999: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
I20260807 20:15:24.804385 16713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.804246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:42979)
0807 20:15:24.804314 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:24.804361 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 20:15:24.804714 16713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:24.804643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:47999)
0807 20:15:24.804680 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:24.804699 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 20:15:24.804808 16711 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 20:15:24.809777 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 16520
I20260807 20:15:25.608783 16715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:25.608539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:54267)
0807 20:15:25.608675 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:25.608717 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260807 20:15:25.682828 16717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:25.682598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:39021)
0807 20:15:25.682731 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:25.682772 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":57,"threads_started":1}
I20260807 20:15:25.805424 16718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:25.805154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:54853)
0807 20:15:25.805313 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:25.805355 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":73,"threads_started":1}
I20260807 20:15:26.609580 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:26.609328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:36513)
0807 20:15:26.609471 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:26.609517 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260807 20:15:26.683686 16720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:26.683433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:33779)
0807 20:15:26.683591 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:26.683641 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
I20260807 20:15:26.806154 16721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:26.805898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:60469)
0807 20:15:26.806071 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:26.806110 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":90,"threads_started":1}
I20260807 20:15:27.610354 16722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:27.610099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:54289)
0807 20:15:27.610262 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:27.610307 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":82,"threads_started":1}
I20260807 20:15:27.684401 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:27.684168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:48073)
0807 20:15:27.684308 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:27.684353 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
I20260807 20:15:27.806944 16725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:27.806673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:44231)
0807 20:15:27.806828 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:27.806872 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260807 20:15:28.611150 16728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:28.610876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:38549)
0807 20:15:28.611053 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:28.611100 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":87,"threads_started":1}
I20260807 20:15:28.685209 16729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:28.684891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:53439)
0807 20:15:28.685067 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:28.685156 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":85,"threads_started":1}
I20260807 20:15:28.807646 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:28.807398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:51139)
0807 20:15:28.807557 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:28.807599 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":80,"threads_started":1}
I20260807 20:15:29.611896 16734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:29.611644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:50555)
0807 20:15:29.611799 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:29.611846 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260807 20:15:29.685945 16738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:29.685707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:37121)
0807 20:15:29.685842 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:29.685890 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260807 20:15:29.808377 16739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:29.808098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:58183)
0807 20:15:29.808288 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:29.808326 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":64,"threads_started":1}
I20260807 20:15:30.612608 16740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:30.612396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:49111)
0807 20:15:30.612521 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:30.612560 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260807 20:15:30.686709 16741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:30.686500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:34885)
0807 20:15:30.686622 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:30.686662 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260807 20:15:30.809041 16742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:30.808841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:43753)
0807 20:15:30.808960 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:30.808996 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
I20260807 20:15:31.613454 16743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:31.613163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:35403)
0807 20:15:31.613326 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:31.613404 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
I20260807 20:15:31.687513 16744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:31.687191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:36987)
0807 20:15:31.687335 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:31.687374 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260807 20:15:31.809710 16745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:31.809478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:54771)
0807 20:15:31.809620 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:31.809662 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260807 20:15:32.614213 16746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:32.613984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:47141)
0807 20:15:32.614115 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:32.614159 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260807 20:15:32.688355 16747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:32.688076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:59853)
0807 20:15:32.688254 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:32.688301 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":78,"threads_started":1}
I20260807 20:15:32.810348 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:32.810141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:57543)
0807 20:15:32.810256 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:32.810297 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
I20260807 20:15:33.614861 16749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:33.614642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:33925)
0807 20:15:33.614776 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:33.614813 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260807 20:15:33.689035 16750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:33.688802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:48087)
0807 20:15:33.688950 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:33.688987 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260807 20:15:33.810993 16755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:33.810782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:33241)
0807 20:15:33.810901 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:33.810945 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260807 20:15:34.615633 16757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:34.615316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:44381)
0807 20:15:34.615516 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:34.615572 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":69,"threads_started":1}
I20260807 20:15:34.689797 16758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:34.689556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:44075)
0807 20:15:34.689713 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:34.689747 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260807 20:15:34.811743 16759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:34.811476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:41407)
0807 20:15:34.811643 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:34.811688 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":81,"threads_started":1}
I20260807 20:15:35.616360 16760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:35.616144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:52197)
0807 20:15:35.616266 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:35.616310 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260807 20:15:35.690402 16764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:35.690196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:43449)
0807 20:15:35.690314 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:35.690352 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260807 20:15:35.812470 16765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:35.812230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:46911)
0807 20:15:35.812360 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:35.812396 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260807 20:15:36.616993 16766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:36.616759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:34871)
0807 20:15:36.616903 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:36.616946 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260807 20:15:36.691141 16767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:36.690856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:45223)
0807 20:15:36.691037 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:36.691081 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":106,"threads_started":1}
I20260807 20:15:36.813208 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:36.812954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:60011)
0807 20:15:36.813115 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:36.813158 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260807 20:15:37.617659 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:37.617449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:33967)
0807 20:15:37.617568 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:37.617609 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260807 20:15:37.691810 16770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:37.691578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:44343)
0807 20:15:37.691705 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:37.691741 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260807 20:15:37.813874 16771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:37.813653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:45599)
0807 20:15:37.813773 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:37.813820 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260807 20:15:38.618309 16773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:38.618074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:32951)
0807 20:15:38.618216 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:38.618260 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":62,"threads_started":1}
I20260807 20:15:38.692503 16775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:38.692270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:51833)
0807 20:15:38.692412 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:38.692452 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":64,"threads_started":1}
I20260807 20:15:38.814595 16777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:38.814308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:45213)
0807 20:15:38.814459 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:38.814504 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":63,"threads_started":1}
I20260807 20:15:39.619174 16778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:39.618908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.193:55579)
0807 20:15:39.619073 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:39.619124 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":60,"threads_started":1}
I20260807 20:15:39.693287 16779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:39.693035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.194:60349)
0807 20:15:39.693199 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:39.693238 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":112,"threads_started":1}
I20260807 20:15:39.812256 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 16044
I20260807 20:15:39.817706 16781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:39.817479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45999 (local address 127.23.200.195:59855)
0807 20:15:39.817629 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:39.817660 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":52,"threads_started":1}
I20260807 20:15:39.821645 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 16315
I20260807 20:15:39.831535 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 16520
2026-08-07T20:15:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18424 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:15:39 dist-test-slave-2j7r krb5kdc[16785](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:15:39 dist-test-slave-2j7r krb5kdc[16785](info): set up 2 sockets
Aug 07 20:15:39 dist-test-slave-2j7r krb5kdc[16785](info): commencing operation
krb5kdc: starting...
W20260807 20:15:41.007268 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:15:41 dist-test-slave-2j7r krb5kdc[16785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:15:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:15:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:15:41.136049 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:39057
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:39057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:41.218539 16802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:41.218744 16802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:41.218765 16802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:41.219880 16802 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 20:15:41.219921 16802 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 20:15:41.219933 16802 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 20:15:41.219942 16802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:15:41.219954 16802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:41.219965 16802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:41.219975 16802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:15:41.219986 16802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:15:41.221472 16802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:39057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:39057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:41.221668 16802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:41.221905 16802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:41.227150 16810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:41.227303 16809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:41.227224 16802 server_base.cc:1034] running on GCE node
W20260807 20:15:41.227159 16812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:41.227607 16802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:41.227905 16802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:41.229076 16802 hybrid_clock.cc:648] HybridClock initialized: now 1786133741229061 us; error 30 us; skew 500 ppm
Aug 07 20:15:41 dist-test-slave-2j7r krb5kdc[16785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133741, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:41.231765 16802 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:15:41.232193 16802 webserver.cc:460] Webserver started at http://127.23.200.254:35451/ using document root <none> and password file <none>
I20260807 20:15:41.232403 16802 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:41.232450 16802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:41.232546 16802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:15:41.233693 16802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "1b155e378287454db55e6d550036c7c5"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "c108ce4cad241ba081286831d85fa702"
server_key_iv: "65c9e786e21516db4bd26a3024e52f42"
server_key_version: "encryptionkey@0"
I20260807 20:15:41.234012 16802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "1b155e378287454db55e6d550036c7c5"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "c108ce4cad241ba081286831d85fa702"
server_key_iv: "65c9e786e21516db4bd26a3024e52f42"
server_key_version: "encryptionkey@0"
I20260807 20:15:41.235364 16802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:15:41.236117 16828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:41.236354 16802 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:15:41.236436 16802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "1b155e378287454db55e6d550036c7c5"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "c108ce4cad241ba081286831d85fa702"
server_key_iv: "65c9e786e21516db4bd26a3024e52f42"
server_key_version: "encryptionkey@0"
I20260807 20:15:41.236503 16802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:41.248643 16802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:41.250633 16802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:41.250764 16802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:41.254642 16802 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:39057
I20260807 20:15:41.254680 16902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:39057 every 8 connection(s)
I20260807 20:15:41.255025 16802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:15:41.255895 16904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:41.258201 16904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Bootstrap starting.
I20260807 20:15:41.258797 16904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:41.259078 16904 log.cc:824] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:41.259750 16904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: No bootstrap required, opened a new log
I20260807 20:15:41.261343 16904 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } }
I20260807 20:15:41.261492 16904 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:41.261535 16904 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b155e378287454db55e6d550036c7c5, State: Initialized, Role: FOLLOWER
I20260807 20:15:41.261646 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 16802
I20260807 20:15:41.261685 16904 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } }
I20260807 20:15:41.261746 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:15:41.261770 16904 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:15:41.261806 16904 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:15:41.261852 16904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:41.261942 24355 external_mini_cluster.cc:1428] Setting key eb22e466870e318aab02421bf2758d28
I20260807 20:15:41.262439 16904 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } }
I20260807 20:15:41.262549 16904 leader_election.cc:302] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b155e378287454db55e6d550036c7c5; no voters:
I20260807 20:15:41.262701 16904 leader_election.cc:288] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:15:41.262918 16910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:15:41.262955 16904 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:15:41.263120 16910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 1 LEADER]: Becoming Leader. State: Replica: 1b155e378287454db55e6d550036c7c5, State: Running, Role: LEADER
I20260807 20:15:41.263276 16910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } }
I20260807 20:15:41.263662 16914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b155e378287454db55e6d550036c7c5. Latest consensus state: current_term: 1 leader_uuid: "1b155e378287454db55e6d550036c7c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } } }
I20260807 20:15:41.263741 16914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:41.264056 16911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b155e378287454db55e6d550036c7c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } } }
I20260807 20:15:41.264111 16911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [sys.catalog]: This master's current role is: LEADER
I20260807 20:15:41.264477 16924 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:15:41.264957 16924 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:15:41.266922 16924 catalog_manager.cc:1347] Generated new cluster ID: d2ddc1a3009848e58e309703d062b19c
I20260807 20:15:41.266976 16924 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:15:41 dist-test-slave-2j7r krb5kdc[16785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:41.271499 16924 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:15:41.272110 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:41.262427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46844 (local address 127.23.200.254:39057)
0807 20:15:41.262609 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:41.262619 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:41.262656 (+ 37us) server_negotiation.cc:407] Connection header received
0807 20:15:41.262780 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:41.262789 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:41.262846 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:41.262993 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:41.263796 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.265196 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.266112 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.266692 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.266808 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:41.268991 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:41.268996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:41.268997 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:41.269008 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:41.270452 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:41.270788 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:41.270791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:41.270791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:41.270890 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:41.271063 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:41.271064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:41.271065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:41.271312 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:41.271494 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:41.271729 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 20:15:41.271955 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":80,"threads_started":1}
I20260807 20:15:41.272149 16924 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:15:41.277999 16924 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Generated new TSK 0
I20260807 20:15:41.278165 16924 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:15:41.307560 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39057
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:41.389876 16941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:41.390080 16941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:41.390111 16941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:41.391232 16941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:41.391270 16941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:41.391305 16941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:15:41.392779 16941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:41.393016 16941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:41.393256 16941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:41.398172 16953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:41.398203 16951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:41.398299 16941 server_base.cc:1034] running on GCE node
W20260807 20:15:41.398200 16955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:41.398571 16941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:41.398846 16941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:41.399997 16941 hybrid_clock.cc:648] HybridClock initialized: now 1786133741399980 us; error 34 us; skew 500 ppm
Aug 07 20:15:41 dist-test-slave-2j7r krb5kdc[16785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133741, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:41.402468 16941 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:15:41.402931 16941 webserver.cc:460] Webserver started at http://127.23.200.193:39743/ using document root <none> and password file <none>
I20260807 20:15:41.403122 16941 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:41.403162 16941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:41.403326 16941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:15:41.404366 16941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "27e9ff6e9b814a7b84940d772a026dbb"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "859365d848584977431a9ced20484524"
server_key_iv: "9c89283a98a997f747741335d644e9a5"
server_key_version: "encryptionkey@0"
I20260807 20:15:41.404661 16941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "27e9ff6e9b814a7b84940d772a026dbb"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "859365d848584977431a9ced20484524"
server_key_iv: "9c89283a98a997f747741335d644e9a5"
server_key_version: "encryptionkey@0"
I20260807 20:15:41.405970 16941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:15:41.406684 16964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:41.406879 16941 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:15:41.406967 16941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "27e9ff6e9b814a7b84940d772a026dbb"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "859365d848584977431a9ced20484524"
server_key_iv: "9c89283a98a997f747741335d644e9a5"
server_key_version: "encryptionkey@0"
I20260807 20:15:41.407037 16941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:41.421343 16941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:41.423617 16941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:41.423753 16941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:41.423990 16941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:41.424293 16941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:15:41.424336 16941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:41.424366 16941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:41.424391 16941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:41.430078 16941 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:34185
I20260807 20:15:41.430132 17151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:34185 every 8 connection(s)
I20260807 20:15:41.430457 16941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:15:41.433357 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 16941
I20260807 20:15:41.433465 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:15:41.433822 24355 external_mini_cluster.cc:1428] Setting key afb94ff26272635d6930b6c70a626f0e
Aug 07 20:15:41 dist-test-slave-2j7r krb5kdc[16785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133741, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:41.441766 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:41.431129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:46301 (local address 127.23.200.254:39057)
0807 20:15:41.431205 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:41.431208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:41.431382 (+ 174us) server_negotiation.cc:407] Connection header received
0807 20:15:41.431847 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:41.431848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:41.431857 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:41.431902 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:41.432848 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.433562 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.435091 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.435346 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.435430 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:41.438611 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:41.438613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:41.438614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:41.438624 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:41.440235 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:41.440711 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:41.440712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:41.440713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:41.440776 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:41.441061 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:41.441062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:41.441063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:41.441230 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:41.441316 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:41.441575 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 20:15:41.441669 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 20:15:41.441890 17154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:41.431094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:46301)
0807 20:15:41.431254 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:41.431292 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:41.431771 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:41.431967 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:41.431970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:41.432550 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:41.432751 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:41.432755 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.433701 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.433702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:41.434976 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:41.434980 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.435545 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.435548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:41.435770 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:41.436660 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:41.436675 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:41.438485 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:41.440328 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:41.440386 (+ 58us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:41.440389 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:41.440634 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:41.440862 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:41.440863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:41.440863 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:41.440994 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:41.441293 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:41.441297 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:41.441447 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 20:15:41.441536 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 20:15:41.441673 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260807 20:15:41.442406 17152 heartbeater.cc:347] Connected to a master server at 127.23.200.254:39057
I20260807 20:15:41.442507 17152 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:41.442739 17152 heartbeater.cc:511] Master 127.23.200.254:39057 requested a full tablet report, sending...
I20260807 20:15:41.443262 16853 ts_manager.cc:194] Registered new tserver with Master: 27e9ff6e9b814a7b84940d772a026dbb (127.23.200.193:34185)
I20260807 20:15:41.443853 16853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:46301
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:15:41.463335 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39057
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:41.545243 17159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:41.545449 17159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:41.545471 17159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:41.546600 17159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:41.546640 17159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:41.546667 17159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:15:41.548122 17159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:41.548344 17159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:41.548575 17159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:41.553381 17174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:41.553524 17177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:41.553392 17173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:41.553627 17159 server_base.cc:1034] running on GCE node
I20260807 20:15:41.553800 17159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:41.554018 17159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:41.555164 17159 hybrid_clock.cc:648] HybridClock initialized: now 1786133741555150 us; error 32 us; skew 500 ppm
Aug 07 20:15:41 dist-test-slave-2j7r krb5kdc[16785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133741, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:41.557557 17159 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:15:41.557935 17159 webserver.cc:460] Webserver started at http://127.23.200.194:43655/ using document root <none> and password file <none>
I20260807 20:15:41.558137 17159 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:41.558175 17159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:41.558264 17159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:15:41.559235 17159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "7b61b5d275cd492aad3c527a4a928d42"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "0a7116fcef202f3df9bb6496fb00a531"
server_key_iv: "eab3bfe1c4fa3f92ac5e0701f1f07699"
server_key_version: "encryptionkey@0"
I20260807 20:15:41.559521 17159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "7b61b5d275cd492aad3c527a4a928d42"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "0a7116fcef202f3df9bb6496fb00a531"
server_key_iv: "eab3bfe1c4fa3f92ac5e0701f1f07699"
server_key_version: "encryptionkey@0"
I20260807 20:15:41.560746 17159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:15:41.561439 17184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:41.561573 17159 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:41.561661 17159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "7b61b5d275cd492aad3c527a4a928d42"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "0a7116fcef202f3df9bb6496fb00a531"
server_key_iv: "eab3bfe1c4fa3f92ac5e0701f1f07699"
server_key_version: "encryptionkey@0"
I20260807 20:15:41.561717 17159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:41.580546 17159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:41.582680 17159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:41.582819 17159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:41.583034 17159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:41.583329 17159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:15:41.583361 17159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:41.583391 17159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:41.583413 17159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:41.588927 17159 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:41909
I20260807 20:15:41.588986 17367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:41909 every 8 connection(s)
I20260807 20:15:41.589313 17159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
Aug 07 20:15:41 dist-test-slave-2j7r krb5kdc[16785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133741, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:41.597886 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:41.590009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:60331 (local address 127.23.200.254:39057)
0807 20:15:41.590066 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:41.590068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:41.590257 (+ 189us) server_negotiation.cc:407] Connection header received
0807 20:15:41.590717 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:41.590718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:41.590728 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:41.590789 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:41.591719 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.592213 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.593181 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.593392 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.593445 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:41.595470 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:41.595471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:41.595472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:41.595479 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:41.596594 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:41.596969 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:41.596970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:41.596970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:41.597024 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:41.597269 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:41.597270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:41.597271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:41.597386 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:41.597444 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:41.597703 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 20:15:41.597794 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 20:15:41.598052 17374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:41.589965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:60331)
0807 20:15:41.590133 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:41.590175 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:41.590643 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:41.590873 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:41.590877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:41.591450 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:41.591646 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:41.591651 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.592282 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.592282 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:41.593077 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:41.593099 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.593455 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.593456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:41.593564 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:41.594031 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:41.594041 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:41.595368 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:41.596671 (+ 1303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:41.596678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:41.596680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:41.596900 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:41.597078 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:41.597079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:41.597080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:41.597212 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:41.597442 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:41.597451 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:41.597585 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 20:15:41.597674 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 20:15:41.597828 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260807 20:15:41.598476 17369 heartbeater.cc:347] Connected to a master server at 127.23.200.254:39057
I20260807 20:15:41.598568 17369 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:41.598659 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 17159
I20260807 20:15:41.598745 17369 heartbeater.cc:511] Master 127.23.200.254:39057 requested a full tablet report, sending...
I20260807 20:15:41.598759 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:15:41.598942 24355 external_mini_cluster.cc:1428] Setting key 205b3cd6c50a0517d3914ebcd12a8f1b
I20260807 20:15:41.599179 16853 ts_manager.cc:194] Registered new tserver with Master: 7b61b5d275cd492aad3c527a4a928d42 (127.23.200.194:41909)
I20260807 20:15:41.599566 16853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:60331
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:15:41.624053 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39057
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:15:41.707302 17380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:15:41.707520 17380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:15:41.707553 17380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:15:41.708726 17380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:15:41.708772 17380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:15:41.708807 17380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:15:41.710403 17380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:39057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:15:41.710659 17380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:15:41.710883 17380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:15:41.716033 17389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:15:41.716032 17392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:41.716185 17380 server_base.cc:1034] running on GCE node
W20260807 20:15:41.716111 17388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:15:41.716529 17380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:15:41.716749 17380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:15:41.717900 17380 hybrid_clock.cc:648] HybridClock initialized: now 1786133741717877 us; error 33 us; skew 500 ppm
Aug 07 20:15:41 dist-test-slave-2j7r krb5kdc[16785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133741, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:15:41.720326 17380 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:15:41.720801 17380 webserver.cc:460] Webserver started at http://127.23.200.195:34901/ using document root <none> and password file <none>
I20260807 20:15:41.721012 17380 fs_manager.cc:359] Metadata directory not provided
I20260807 20:15:41.721061 17380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:15:41.721215 17380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:15:41.722235 17380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "eb5b61d194c0458cb991dc85df1590c0"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "8894b0d59f0664516b3d3788d9ce5a76"
server_key_iv: "7b0e46a947fe53dae6203ae0610960db"
server_key_version: "encryptionkey@0"
I20260807 20:15:41.722563 17380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "eb5b61d194c0458cb991dc85df1590c0"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "8894b0d59f0664516b3d3788d9ce5a76"
server_key_iv: "7b0e46a947fe53dae6203ae0610960db"
server_key_version: "encryptionkey@0"
I20260807 20:15:41.723887 17380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:15:41.724581 17404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:41.724738 17380 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:15:41.724829 17380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "eb5b61d194c0458cb991dc85df1590c0"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "8894b0d59f0664516b3d3788d9ce5a76"
server_key_iv: "7b0e46a947fe53dae6203ae0610960db"
server_key_version: "encryptionkey@0"
I20260807 20:15:41.724901 17380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:15:41.737547 17380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:15:41.739537 17380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:15:41.739674 17380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:15:41.739884 17380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:15:41.740196 17380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:15:41.740235 17380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:41.740264 17380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:15:41.740288 17380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:15:41.746045 17380 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:35053
I20260807 20:15:41.746094 17603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:35053 every 8 connection(s)
I20260807 20:15:41.746434 17380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:15:41.749847 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 17380
I20260807 20:15:41.749967 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:15:41.750170 24355 external_mini_cluster.cc:1428] Setting key a2be9affb52c4e7b41171da2f3e4705c
Aug 07 20:15:41 dist-test-slave-2j7r krb5kdc[16785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133741, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:15:41.755756 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:41.747107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:39287 (local address 127.23.200.254:39057)
0807 20:15:41.747161 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:41.747163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:41.747356 (+ 193us) server_negotiation.cc:407] Connection header received
0807 20:15:41.747851 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:41.747852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:41.747866 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:41.747919 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:41.748915 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.749688 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.751271 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.751474 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.751526 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:41.753574 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:41.753575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:41.753575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:41.753583 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:41.754673 (+ 1090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:41.755021 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:41.755021 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:41.755022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:41.755067 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:41.755251 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:41.755251 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:41.755252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:41.755380 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:41.755431 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:41.755632 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 20:15:41.755691 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 20:15:41.755937 17606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:41.747075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:39287)
0807 20:15:41.747225 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:41.747266 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:41.747747 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:41.747990 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:41.747994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:41.748622 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:15:41.748814 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:41.748819 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.749833 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.749835 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:41.751192 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:41.751195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.751540 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.751541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:41.751663 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:41.752100 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:15:41.752109 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:15:41.753481 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:15:41.754742 (+ 1261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:41.754749 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:41.754751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:41.754965 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:41.755114 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:15:41.755114 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:15:41.755114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:15:41.755209 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:15:41.755426 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:15:41.755429 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:15:41.755537 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 20:15:41.755613 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 20:15:41.755729 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260807 20:15:41.756374 17604 heartbeater.cc:347] Connected to a master server at 127.23.200.254:39057
I20260807 20:15:41.756471 17604 heartbeater.cc:464] Registering TS with master...
I20260807 20:15:41.756649 17604 heartbeater.cc:511] Master 127.23.200.254:39057 requested a full tablet report, sending...
I20260807 20:15:41.757021 16852 ts_manager.cc:194] Registered new tserver with Master: eb5b61d194c0458cb991dc85df1590c0 (127.23.200.195:35053)
I20260807 20:15:41.757405 16852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:39287
I20260807 20:15:41.762142 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:15:41.769438 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:41.763569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46850 (local address 127.23.200.254:39057)
0807 20:15:41.763627 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:41.763629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:41.763705 (+ 76us) server_negotiation.cc:407] Connection header received
0807 20:15:41.763747 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:41.763748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:41.763756 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:41.763797 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:15:41.764403 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.764965 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.765797 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.766008 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.766060 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:41.767112 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:15:41.767113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:15:41.767114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:15:41.767121 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:15:41.768319 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:41.768661 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:41.768662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:41.768662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:41.768720 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:15:41.768882 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:15:41.768883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:15:41.768884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:15:41.769031 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:15:41.769140 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:41.769270 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 20:15:41.769348 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 20:15:41.770900 16852 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:15:41.772615 16852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:15:41.778923 17615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:41.774647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:35053 (local address 127.0.0.1:51838)
0807 20:15:41.774985 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:41.775028 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:41.775095 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:41.775330 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:41.775333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:41.775336 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:41.775404 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:41.775406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.776693 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.776694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:41.777807 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:41.777810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.778589 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.778590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:41.778679 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:41.778697 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 20:15:41.778762 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 20:15:41.778815 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":73,"thread_start_us":53,"threads_started":1}
I20260807 20:15:41.779035 17619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:41.774782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49906 (local address 127.23.200.194:41909)
0807 20:15:41.775201 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:41.775205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:41.775214 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:15:41.775247 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:41.775249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:41.775265 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:41.775341 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:41.775663 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.776693 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.778043 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.778682 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.778764 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:41.778821 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:41.778915 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 20:15:41.778955 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":95,"threads_started":1}
I20260807 20:15:41.779354 17623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:41.774967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47826 (local address 127.23.200.193:34185)
0807 20:15:41.775147 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:41.775150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:41.775662 (+ 512us) server_negotiation.cc:407] Connection header received
0807 20:15:41.775701 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:41.775702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:41.775715 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:41.775775 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:41.775884 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.776552 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.778194 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.778870 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.778929 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:41.778984 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:41.779262 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 20:15:41.779298 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":95,"threads_started":1}
I20260807 20:15:41.779389 17622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:41.774829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51838 (local address 127.23.200.195:35053)
0807 20:15:41.775018 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:41.775023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:41.775171 (+ 148us) server_negotiation.cc:407] Connection header received
0807 20:15:41.775219 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:41.775222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:41.775238 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:41.775319 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:41.775464 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.776581 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.777881 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.778494 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.778565 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:41.778620 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:41.779271 (+ 651us) server_negotiation.cc:299] Negotiation successful
0807 20:15:41.779325 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":86,"threads_started":1}
I20260807 20:15:41.779028 17614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:41.774647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:41909 (local address 127.0.0.1:49906)
0807 20:15:41.774995 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:41.775029 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:41.775094 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:41.775554 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:41.775554 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:41.775556 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:41.775607 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:41.775609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.776780 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.776781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:41.777952 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:41.777955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.778772 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.778773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:41.778837 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:41.778842 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:15:41.778902 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 20:15:41.778953 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":5,"thread_start_us":55,"threads_started":1}
I20260807 20:15:41.779211 17620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:41.774868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:34185 (local address 127.0.0.1:47826)
0807 20:15:41.775441 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:41.775450 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:41.775501 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:41.775783 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:41.775784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:41.775786 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:41.775821 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:41.775823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.776638 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.776639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:41.778117 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:41.778119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:41.779017 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:41.779018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:41.779072 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:41.779075 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:15:41.779122 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 20:15:41.779156 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":82,"thread_start_us":37,"threads_started":1}
I20260807 20:15:41.780182 17482 tablet_service.cc:1467] Processing CreateTablet for tablet d25cfe0de97c425fab7042c0d1af0810 (DEFAULT_TABLE table=test-table [id=5751f5291c0542f4891731b673d6ed86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:15:41.780210 17255 tablet_service.cc:1467] Processing CreateTablet for tablet d25cfe0de97c425fab7042c0d1af0810 (DEFAULT_TABLE table=test-table [id=5751f5291c0542f4891731b673d6ed86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:15:41.780336 17034 tablet_service.cc:1467] Processing CreateTablet for tablet d25cfe0de97c425fab7042c0d1af0810 (DEFAULT_TABLE table=test-table [id=5751f5291c0542f4891731b673d6ed86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:15:41.780480 17482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25cfe0de97c425fab7042c0d1af0810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:41.780550 17255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25cfe0de97c425fab7042c0d1af0810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:41.780606 17034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25cfe0de97c425fab7042c0d1af0810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:15:41.783124 17624 tablet_bootstrap.cc:492] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0: Bootstrap starting.
I20260807 20:15:41.783893 17624 tablet_bootstrap.cc:654] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:41.783921 17629 tablet_bootstrap.cc:492] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: Bootstrap starting.
I20260807 20:15:41.784402 17624 log.cc:824] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:41.784533 17629 tablet_bootstrap.cc:654] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:41.785030 17629 log.cc:824] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:41.785179 17624 tablet_bootstrap.cc:492] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0: No bootstrap required, opened a new log
I20260807 20:15:41.785249 17624 ts_tablet_manager.cc:1397] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:15:41.785252 17625 tablet_bootstrap.cc:492] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42: Bootstrap starting.
I20260807 20:15:41.785851 17625 tablet_bootstrap.cc:654] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42: Neither blocks nor log segments found. Creating new log.
I20260807 20:15:41.786319 17625 log.cc:824] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42: Log is configured to *not* fsync() on all Append() calls
I20260807 20:15:41.786965 17625 tablet_bootstrap.cc:492] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42: No bootstrap required, opened a new log
I20260807 20:15:41.787017 17625 ts_tablet_manager.cc:1397] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:15:41.786899 17624 raft_consensus.cc:348] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:15:41.787047 17624 raft_consensus.cc:374] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:41.787091 17624 raft_consensus.cc:729] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5b61d194c0458cb991dc85df1590c0, State: Initialized, Role: FOLLOWER
I20260807 20:15:41.787201 17624 consensus_queue.cc:260] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:15:41.787459 17624 ts_tablet_manager.cc:1428] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:15:41.787477 17604 heartbeater.cc:503] Master 127.23.200.254:39057 was elected leader, sending a full tablet report...
I20260807 20:15:41.788599 17625 raft_consensus.cc:348] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:15:41.788757 17625 raft_consensus.cc:374] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:41.788789 17625 raft_consensus.cc:729] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b61b5d275cd492aad3c527a4a928d42, State: Initialized, Role: FOLLOWER
I20260807 20:15:41.788869 17625 consensus_queue.cc:260] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:15:41.789077 17625 ts_tablet_manager.cc:1428] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:15:41.789152 17369 heartbeater.cc:503] Master 127.23.200.254:39057 was elected leader, sending a full tablet report...
I20260807 20:15:41.789505 17629 tablet_bootstrap.cc:492] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: No bootstrap required, opened a new log
I20260807 20:15:41.789592 17629 ts_tablet_manager.cc:1397] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260807 20:15:41.791191 17629 raft_consensus.cc:348] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:15:41.791373 17629 raft_consensus.cc:374] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:15:41.791412 17629 raft_consensus.cc:729] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27e9ff6e9b814a7b84940d772a026dbb, State: Initialized, Role: FOLLOWER
I20260807 20:15:41.791520 17629 consensus_queue.cc:260] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:15:41.791746 17629 ts_tablet_manager.cc:1428] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:15:41.791782 17152 heartbeater.cc:503] Master 127.23.200.254:39057 was elected leader, sending a full tablet report...
W20260807 20:15:41.839862 17370 tablet.cc:2369] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:15:41.931174 17153 tablet.cc:2369] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:15:41.997030 17605 tablet.cc:2369] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:15:42.023101 17638 raft_consensus.cc:482] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:15:42.023281 17638 raft_consensus.cc:504] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:15:42.023694 17638 leader_election.cc:288] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b61b5d275cd492aad3c527a4a928d42 (127.23.200.194:41909), eb5b61d194c0458cb991dc85df1590c0 (127.23.200.195:35053)
I20260807 20:15:42.026957 17619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:42.024047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:49347 (local address 127.23.200.194:41909)
0807 20:15:42.024150 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:42.024152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:42.024346 (+ 194us) server_negotiation.cc:407] Connection header received
0807 20:15:42.024376 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:42.024377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:42.024386 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:42.024427 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:42.024588 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:42.025302 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:42.026280 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:42.026653 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:42.026715 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:42.026724 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:42.026849 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 20:15:42.026881 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 20:15:42.027001 17639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:42.023882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:41909 (local address 127.23.200.193:49347)
0807 20:15:42.024255 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:42.024263 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:42.024303 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:42.024467 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:42.024467 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:42.024468 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:42.024519 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:42.024521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:42.025376 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:42.025377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:42.026195 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:42.026198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:42.026721 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:42.026722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:42.026791 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:42.026795 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:15:42.026839 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 20:15:42.026879 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":27,"thread_start_us":98,"threads_started":1}
I20260807 20:15:42.027001 17622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:42.024016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:40567 (local address 127.23.200.195:35053)
0807 20:15:42.024064 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:42.024066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:42.024165 (+ 99us) server_negotiation.cc:407] Connection header received
0807 20:15:42.024239 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:42.024240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:42.024250 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:42.024316 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:15:42.024579 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:42.025300 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:42.026290 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:42.026668 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:42.026731 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:42.026739 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:42.026870 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 20:15:42.026908 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:15:42.027006 17640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:42.023878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:35053 (local address 127.23.200.193:40567)
0807 20:15:42.024109 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:42.024120 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:15:42.024182 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:15:42.024370 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:15:42.024371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:15:42.024373 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:15:42.024458 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:42.024460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:42.025410 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:42.025411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:42.026200 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:15:42.026202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:15:42.026741 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:15:42.026742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:15:42.026807 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:42.026810 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:15:42.026855 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 20:15:42.026895 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":105,"threads_started":1}
I20260807 20:15:42.027413 17292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25cfe0de97c425fab7042c0d1af0810" candidate_uuid: "27e9ff6e9b814a7b84940d772a026dbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b61b5d275cd492aad3c527a4a928d42" is_pre_election: true
I20260807 20:15:42.027568 17292 raft_consensus.cc:2454] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27e9ff6e9b814a7b84940d772a026dbb in term 0.
I20260807 20:15:42.027791 16965 leader_election.cc:302] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27e9ff6e9b814a7b84940d772a026dbb, 7b61b5d275cd492aad3c527a4a928d42; no voters:
I20260807 20:15:42.027935 17638 raft_consensus.cc:2781] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:15:42.027897 17519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25cfe0de97c425fab7042c0d1af0810" candidate_uuid: "27e9ff6e9b814a7b84940d772a026dbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5b61d194c0458cb991dc85df1590c0" is_pre_election: true
I20260807 20:15:42.027984 17638 raft_consensus.cc:482] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:15:42.028012 17638 raft_consensus.cc:3037] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:42.028044 17519 raft_consensus.cc:2454] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27e9ff6e9b814a7b84940d772a026dbb in term 0.
I20260807 20:15:42.028762 17638 raft_consensus.cc:504] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:15:42.028892 17638 leader_election.cc:288] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 1 election: Requested vote from peers 7b61b5d275cd492aad3c527a4a928d42 (127.23.200.194:41909), eb5b61d194c0458cb991dc85df1590c0 (127.23.200.195:35053)
I20260807 20:15:42.029027 17292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25cfe0de97c425fab7042c0d1af0810" candidate_uuid: "27e9ff6e9b814a7b84940d772a026dbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b61b5d275cd492aad3c527a4a928d42"
I20260807 20:15:42.029058 17519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25cfe0de97c425fab7042c0d1af0810" candidate_uuid: "27e9ff6e9b814a7b84940d772a026dbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5b61d194c0458cb991dc85df1590c0"
I20260807 20:15:42.029120 17292 raft_consensus.cc:3037] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:42.029151 17519 raft_consensus.cc:3037] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:15:42.029839 17292 raft_consensus.cc:2454] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27e9ff6e9b814a7b84940d772a026dbb in term 1.
I20260807 20:15:42.029857 17519 raft_consensus.cc:2454] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27e9ff6e9b814a7b84940d772a026dbb in term 1.
I20260807 20:15:42.030025 16965 leader_election.cc:302] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27e9ff6e9b814a7b84940d772a026dbb, 7b61b5d275cd492aad3c527a4a928d42; no voters:
I20260807 20:15:42.030119 17638 raft_consensus.cc:2781] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:15:42.030282 17638 raft_consensus.cc:686] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 LEADER]: Becoming Leader. State: Replica: 27e9ff6e9b814a7b84940d772a026dbb, State: Running, Role: LEADER
I20260807 20:15:42.030391 17638 consensus_queue.cc:237] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:15:42.031208 16852 catalog_manager.cc:5495] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb reported cstate change: term changed from 0 to 1, leader changed from <none> to 27e9ff6e9b814a7b84940d772a026dbb (127.23.200.193). New cstate: current_term: 1 leader_uuid: "27e9ff6e9b814a7b84940d772a026dbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } health_report { overall_health: HEALTHY } } }
I20260807 20:15:42.060640 17623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:42.058350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47838 (local address 127.23.200.193:34185)
0807 20:15:42.058437 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 20:15:42.058439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:15:42.058447 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:15:42.058484 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:15:42.058485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:15:42.058503 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:15:42.058562 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:15:42.058711 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:42.059174 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:42.059910 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:15:42.060096 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:15:42.060150 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:15:42.060309 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:15:42.060425 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:15:42.060500 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 20:15:42.060538 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 20:15:42.060559 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:15:42.062439 17519 raft_consensus.cc:1261] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0 [term 1 FOLLOWER]: Refusing update from remote peer 27e9ff6e9b814a7b84940d772a026dbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:15:42.062453 17292 raft_consensus.cc:1261] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42 [term 1 FOLLOWER]: Refusing update from remote peer 27e9ff6e9b814a7b84940d772a026dbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:15:42.062821 17641 consensus_queue.cc:1035] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:15:42.062949 17638 consensus_queue.cc:1035] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:15:42.065284 17647 mvcc.cc:205] Tried to move back new op lower bound from 7316003807485140992 to 7316003807357046784. Current Snapshot: MvccSnapshot[applied={T|T < 7316003807485140992}]
I20260807 20:15:42.066001 17648 mvcc.cc:205] Tried to move back new op lower bound from 7316003807485140992 to 7316003807357046784. Current Snapshot: MvccSnapshot[applied={T|T < 7316003807485140992 or (T in {7316003807485140992})}]
I20260807 20:15:42.066116 17649 mvcc.cc:205] Tried to move back new op lower bound from 7316003807485140992 to 7316003807357046784. Current Snapshot: MvccSnapshot[applied={T|T < 7316003807485140992}]
I20260807 20:15:42.067575 16851 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:15:42.067658 16851 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:15:42.068364 16851 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Sending DeleteTablet for 3 replicas of tablet d25cfe0de97c425fab7042c0d1af0810
I20260807 20:15:42.068682 17034 tablet_service.cc:1514] Processing DeleteTablet for tablet d25cfe0de97c425fab7042c0d1af0810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:15:42 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:47826
I20260807 20:15:42.068682 17255 tablet_service.cc:1514] Processing DeleteTablet for tablet d25cfe0de97c425fab7042c0d1af0810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:15:42 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:49906
I20260807 20:15:42.068727 17482 tablet_service.cc:1514] Processing DeleteTablet for tablet d25cfe0de97c425fab7042c0d1af0810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:15:42 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:51838
I20260807 20:15:42.068913 17661 tablet_replica.cc:330] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: stopping tablet replica
I20260807 20:15:42.069010 17667 tablet_replica.cc:330] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0: stopping tablet replica
I20260807 20:15:42.068997 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 16941
I20260807 20:15:42.069077 17660 tablet_replica.cc:330] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42: stopping tablet replica
I20260807 20:15:42.069185 17667 raft_consensus.cc:2229] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:15:42.069296 17660 raft_consensus.cc:2229] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:15:42.069325 17667 raft_consensus.cc:2258] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:15:42.069371 17661 raft_consensus.cc:2229] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:15:42.069571 17660 raft_consensus.cc:2258] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:15:42.072000 17660 ts_tablet_manager.cc:1905] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:15:42.073949 17660 ts_tablet_manager.cc:1918] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:15:42.074041 17660 log.cc:1196] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/d25cfe0de97c425fab7042c0d1af0810
I20260807 20:15:42.074328 17660 ts_tablet_manager.cc:1939] T d25cfe0de97c425fab7042c0d1af0810 P 7b61b5d275cd492aad3c527a4a928d42: Deleting consensus metadata
I20260807 20:15:42.074882 16829 catalog_manager.cc:4841] TS 7b61b5d275cd492aad3c527a4a928d42 (127.23.200.194:41909): tablet d25cfe0de97c425fab7042c0d1af0810 (table test-table [id=5751f5291c0542f4891731b673d6ed86]) successfully deleted
I20260807 20:15:42.080610 17667 ts_tablet_manager.cc:1905] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:15:42.082290 17667 ts_tablet_manager.cc:1918] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:15:42.082369 17667 log.cc:1196] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/d25cfe0de97c425fab7042c0d1af0810
I20260807 20:15:42.082590 17667 ts_tablet_manager.cc:1939] T d25cfe0de97c425fab7042c0d1af0810 P eb5b61d194c0458cb991dc85df1590c0: Deleting consensus metadata
I20260807 20:15:42.083024 16832 catalog_manager.cc:4841] TS eb5b61d194c0458cb991dc85df1590c0 (127.23.200.195:35053): tablet d25cfe0de97c425fab7042c0d1af0810 (table test-table [id=5751f5291c0542f4891731b673d6ed86]) successfully deleted
W20260807 20:15:42.086359 16829 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:34185 (error 108)
I20260807 20:15:42.086627 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 17159
I20260807 20:15:42.090080 17620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:15:42.086709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:34185 (local address 127.0.0.1:47844)
0807 20:15:42.089985 (+ 3276us) negotiation.cc:107] Waiting for socket to connect
0807 20:15:42.090040 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:34185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3245}
W20260807 20:15:42.093539 16829 catalog_manager.cc:4584] TS 27e9ff6e9b814a7b84940d772a026dbb (127.23.200.193:34185): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d25cfe0de97c425fab7042c0d1af0810: Network error: Client connection negotiation failed: client connection to 127.23.200.193:34185: connect: Connection refused (error 111)
I20260807 20:15:42.097030 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 17380
I20260807 20:15:42.104182 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 16802
I20260807 20:16:02.117949 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:39057
--webserver_interface=127.23.200.254
--webserver_port=35451
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:39057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:02.202785 17669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:02.203032 17669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:02.203054 17669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:02.204200 17669 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 20:16:02.204236 17669 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 20:16:02.204248 17669 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 20:16:02.204259 17669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:16:02.204272 17669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:02.204283 17669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:02.204293 17669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:16:02.204304 17669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:16:02.205870 17669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:39057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:39057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=35451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:02.206122 17669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:02.206310 17669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:02.215752 17678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:02.215814 17681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:02.215945 17669 server_base.cc:1034] running on GCE node
W20260807 20:16:02.215803 17676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:02.216244 17669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:02.216553 17669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:02.217693 17669 hybrid_clock.cc:648] HybridClock initialized: now 1786133762217682 us; error 33 us; skew 500 ppm
Aug 07 20:16:02 dist-test-slave-2j7r krb5kdc[16785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133762, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:02.220253 17669 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:16:02.220738 17669 webserver.cc:460] Webserver started at http://127.23.200.254:35451/ using document root <none> and password file <none>
I20260807 20:16:02.220995 17669 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:02.221076 17669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:02.222476 17669 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:16:02.223169 17698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:02.223325 17669 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:16:02.223416 17669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "1b155e378287454db55e6d550036c7c5"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "c108ce4cad241ba081286831d85fa702"
server_key_iv: "65c9e786e21516db4bd26a3024e52f42"
server_key_version: "encryptionkey@0"
I20260807 20:16:02.223820 17669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:02.243557 17669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:02.245625 17669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:02.245774 17669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:02.249995 17669 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:39057
I20260807 20:16:02.250054 17778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:39057 every 8 connection(s)
I20260807 20:16:02.250445 17669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:16:02.251995 17780 sys_catalog.cc:263] Verifying existing consensus state
I20260807 20:16:02.252727 17780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Bootstrap starting.
I20260807 20:16:02.254629 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 17669
I20260807 20:16:02.254899 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:34185
--local_ip_for_outbound_sockets=127.23.200.193
--tserver_master_addrs=127.23.200.254:39057
--webserver_port=39743
--webserver_interface=127.23.200.193
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:16:02.261354 17780 log.cc:824] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:02.263397 17780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:16:02.263671 17780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Bootstrap complete.
I20260807 20:16:02.265921 17780 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } }
I20260807 20:16:02.266170 17780 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b155e378287454db55e6d550036c7c5, State: Initialized, Role: FOLLOWER
I20260807 20:16:02.266331 17780 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } }
I20260807 20:16:02.266400 17780 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:16:02.266459 17780 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:16:02.266510 17780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 20:16:02.267534 17780 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } }
I20260807 20:16:02.267633 17780 leader_election.cc:302] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b155e378287454db55e6d550036c7c5; no voters:
I20260807 20:16:02.267807 17780 leader_election.cc:288] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 20:16:02.267855 17786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 20:16:02.267992 17786 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 2 LEADER]: Becoming Leader. State: Replica: 1b155e378287454db55e6d550036c7c5, State: Running, Role: LEADER
I20260807 20:16:02.268082 17780 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:16:02.268153 17786 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } }
I20260807 20:16:02.268461 17788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b155e378287454db55e6d550036c7c5. Latest consensus state: current_term: 2 leader_uuid: "1b155e378287454db55e6d550036c7c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } } }
I20260807 20:16:02.268477 17787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1b155e378287454db55e6d550036c7c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } } }
I20260807 20:16:02.268630 17787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:02.268831 17793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:16:02.268883 17788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:02.269819 17793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:16:02.269940 17793 catalog_manager.cc:1259] Loaded cluster ID: d2ddc1a3009848e58e309703d062b19c
I20260807 20:16:02.270021 17793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:16:02.271296 17793 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:16:02.271481 17793 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Loaded TSK: 0
I20260807 20:16:02.291234 17793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Generated new TSK 1
I20260807 20:16:02.291347 17793 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 20:16:02.339423 17782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:02.339648 17782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:02.339681 17782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:02.340904 17782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:02.340953 17782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:02.340991 17782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:16:02.342520 17782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:34185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=39743
--tserver_master_addrs=127.23.200.254:39057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:02.342758 17782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:02.342988 17782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:02.348057 17824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:02.348078 17823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:02.348102 17829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:02.348456 17782 server_base.cc:1034] running on GCE node
I20260807 20:16:02.348680 17782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:02.348992 17782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:02.350180 17782 hybrid_clock.cc:648] HybridClock initialized: now 1786133762350159 us; error 34 us; skew 500 ppm
Aug 07 20:16:02 dist-test-slave-2j7r krb5kdc[16785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133762, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:02.353332 17782 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:16:02.353946 17782 webserver.cc:460] Webserver started at http://127.23.200.193:39743/ using document root <none> and password file <none>
I20260807 20:16:02.354230 17782 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:02.354318 17782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:02.356096 17782 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:16:02.356889 17839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:02.357069 17782 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:16:02.357172 17782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "27e9ff6e9b814a7b84940d772a026dbb"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "859365d848584977431a9ced20484524"
server_key_iv: "9c89283a98a997f747741335d644e9a5"
server_key_version: "encryptionkey@0"
I20260807 20:16:02.357543 17782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:02.379347 17782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:02.381614 17782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:02.381742 17782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:02.381991 17782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:02.382448 17852 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 20:16:02.384492 17782 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 20:16:02.384544 17782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 20:16:02.384580 17782 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 20:16:02.385272 17782 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 20:16:02.385306 17782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:02.385401 17852 tablet_bootstrap.cc:492] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: Bootstrap starting.
I20260807 20:16:02.392477 17782 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:34185
I20260807 20:16:02.392556 18025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:34185 every 8 connection(s)
I20260807 20:16:02.392917 17782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:16:02.396837 17852 log.cc:824] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:02.399590 17852 tablet_bootstrap.cc:492] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:16:02.399978 17852 tablet_bootstrap.cc:492] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: Bootstrap complete.
I20260807 20:16:02.400796 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 17782
I20260807 20:16:02.401108 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:41909
--local_ip_for_outbound_sockets=127.23.200.194
--tserver_master_addrs=127.23.200.254:39057
--webserver_port=43655
--webserver_interface=127.23.200.194
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 20:16:02 dist-test-slave-2j7r krb5kdc[16785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133762, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:16:02.401235 17852 ts_tablet_manager.cc:1397] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.010s
I20260807 20:16:02.402526 17852 raft_consensus.cc:348] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:02.402837 17852 raft_consensus.cc:729] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27e9ff6e9b814a7b84940d772a026dbb, State: Initialized, Role: FOLLOWER
I20260807 20:16:02.403010 17852 consensus_queue.cc:260] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:02.403263 17852 ts_tablet_manager.cc:1428] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:16:02.405383 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.393746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:56537 (local address 127.23.200.254:39057)
0807 20:16:02.395787 (+ 2041us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:02.395796 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:02.395830 (+ 34us) server_negotiation.cc:407] Connection header received
0807 20:16:02.395966 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:02.395972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:02.396016 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:02.396147 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:02.396808 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.397918 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.398919 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.399315 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.399421 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.401798 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:02.401803 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:02.401804 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:02.401817 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:02.403727 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:02.404137 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:02.404141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:02.404142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:02.404239 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:02.404466 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:02.404467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:02.404468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:02.404695 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:02.404769 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:02.405043 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 20:16:02.405192 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1979,"thread_start_us":85,"threads_started":1}
I20260807 20:16:02.405383 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.393648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:56537)
0807 20:16:02.393848 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:02.393897 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:02.394358 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:02.396196 (+ 1838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:02.396201 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:02.396491 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:02.396683 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:02.396689 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.398038 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.398039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:02.398829 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:02.398832 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.399420 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.399422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:02.399549 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.400053 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:16:02.400065 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:02.401682 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:02.403803 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:02.403811 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:02.403813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:02.404058 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:02.404300 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:02.404301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:02.404302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:02.404408 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:02.404759 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:02.404766 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:02.404887 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 20:16:02.404977 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 20:16:02.405178 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
I20260807 20:16:02.406301 18026 heartbeater.cc:347] Connected to a master server at 127.23.200.254:39057
I20260807 20:16:02.406404 18026 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:02.406574 18026 heartbeater.cc:511] Master 127.23.200.254:39057 requested a full tablet report, sending...
I20260807 20:16:02.407289 17729 ts_manager.cc:194] Registered new tserver with Master: 27e9ff6e9b814a7b84940d772a026dbb (127.23.200.193:34185)
I20260807 20:16:02.408317 17729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:56537
I20260807 20:16:02.409169 18026 heartbeater.cc:503] Master 127.23.200.254:39057 was elected leader, sending a full tablet report...
I20260807 20:16:02.415786 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.408101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56558 (local address 127.23.200.193:34185)
0807 20:16:02.408263 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:02.408267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:02.412468 (+ 4201us) server_negotiation.cc:407] Connection header received
0807 20:16:02.412561 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:02.412563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:02.412576 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:02.412648 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:02.412831 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.413542 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.415019 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.415456 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.415515 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.415550 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:02.415707 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 20:16:02.415733 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260807 20:16:02.415818 18038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.407849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:34185 (local address 127.0.0.1:56558)
0807 20:16:02.412310 (+ 4461us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:02.412356 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:02.412463 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:02.412684 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:02.412687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:02.412690 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:02.412765 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:02.412768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.413621 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.413622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:02.414933 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:02.414937 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.415530 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.415531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:02.415611 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.415624 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 20:16:02.415687 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 20:16:02.415740 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4427,"thread_start_us":88,"threads_started":1}
I20260807 20:16:02.416177 17914 tablet_service.cc:1514] Processing DeleteTablet for tablet d25cfe0de97c425fab7042c0d1af0810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:15:42 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:56558
I20260807 20:16:02.416648 18040 tablet_replica.cc:330] stopping tablet replica
I20260807 20:16:02.416746 18040 raft_consensus.cc:2229] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:16:02.416819 18040 raft_consensus.cc:2258] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:02.417223 18040 ts_tablet_manager.cc:1905] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:16:02.419291 18040 ts_tablet_manager.cc:1918] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:16:02.419355 18040 log.cc:1196] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal/wals/d25cfe0de97c425fab7042c0d1af0810
I20260807 20:16:02.419591 18040 ts_tablet_manager.cc:1939] T d25cfe0de97c425fab7042c0d1af0810 P 27e9ff6e9b814a7b84940d772a026dbb: Deleting consensus metadata
I20260807 20:16:02.420102 17700 catalog_manager.cc:4841] TS 27e9ff6e9b814a7b84940d772a026dbb (127.23.200.193:34185): tablet d25cfe0de97c425fab7042c0d1af0810 (table test-table [id=5751f5291c0542f4891731b673d6ed86]) successfully deleted
W20260807 20:16:02.488406 18036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:02.488600 18036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:02.488621 18036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:02.489711 18036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:02.489746 18036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:02.489771 18036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:16:02.491189 18036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:41909
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=43655
--tserver_master_addrs=127.23.200.254:39057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:02.491390 18036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:02.491569 18036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:02.496507 18051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:02.496642 18047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:02.496518 18048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:02.496668 18036 server_base.cc:1034] running on GCE node
I20260807 20:16:02.496933 18036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:02.497189 18036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:02.498338 18036 hybrid_clock.cc:648] HybridClock initialized: now 1786133762498320 us; error 30 us; skew 500 ppm
Aug 07 20:16:02 dist-test-slave-2j7r krb5kdc[16785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133762, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:02.500758 18036 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:16:02.501279 18036 webserver.cc:460] Webserver started at http://127.23.200.194:43655/ using document root <none> and password file <none>
I20260807 20:16:02.501513 18036 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:02.501579 18036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:02.502841 18036 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:02.503368 18058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:02.503512 18036 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:16:02.503607 18036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "7b61b5d275cd492aad3c527a4a928d42"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "0a7116fcef202f3df9bb6496fb00a531"
server_key_iv: "eab3bfe1c4fa3f92ac5e0701f1f07699"
server_key_version: "encryptionkey@0"
I20260807 20:16:02.503934 18036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:02.515195 18036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:02.516835 18036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:02.516971 18036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:02.517200 18036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:02.517493 18036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:02.517525 18036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:02.517553 18036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:02.517576 18036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:02.523169 18036 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:41909
I20260807 20:16:02.523226 18253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:41909 every 8 connection(s)
I20260807 20:16:02.523545 18036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:16:02.528434 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 18036
I20260807 20:16:02.528766 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:35053
--local_ip_for_outbound_sockets=127.23.200.195
--tserver_master_addrs=127.23.200.254:39057
--webserver_port=34901
--webserver_interface=127.23.200.195
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 20:16:02 dist-test-slave-2j7r krb5kdc[16785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133762, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:16:02.533311 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.524219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:33431 (local address 127.23.200.254:39057)
0807 20:16:02.524282 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:02.524284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:02.524501 (+ 217us) server_negotiation.cc:407] Connection header received
0807 20:16:02.524954 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:02.524955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:02.524965 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:02.525019 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:02.525983 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.526520 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.527611 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.527816 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.527870 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.530372 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:02.530374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:02.530374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:02.530381 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:02.531801 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:02.532232 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:02.532233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:02.532234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:02.532287 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:02.532604 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:02.532606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:02.532606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:02.532730 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:02.532812 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:02.533159 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 20:16:02.533243 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:16:02.533569 18257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.524211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:33431)
0807 20:16:02.524366 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:02.524409 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:02.524871 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:02.525073 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:02.525077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:02.525701 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:02.525892 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:02.525907 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.526624 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.526625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:02.527521 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:02.527525 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.527888 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.527889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:02.527991 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.528526 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:16:02.528535 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:02.530206 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:02.531883 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:02.531890 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:02.531892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:02.532167 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:02.532345 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:02.532346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:02.532346 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:02.532492 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:02.532813 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:02.532820 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:02.532981 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 20:16:02.533130 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 20:16:02.533338 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260807 20:16:02.534112 18255 heartbeater.cc:347] Connected to a master server at 127.23.200.254:39057
I20260807 20:16:02.534221 18255 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:02.534435 18255 heartbeater.cc:511] Master 127.23.200.254:39057 requested a full tablet report, sending...
I20260807 20:16:02.534822 17729 ts_manager.cc:194] Registered new tserver with Master: 7b61b5d275cd492aad3c527a4a928d42 (127.23.200.194:41909)
I20260807 20:16:02.535194 17729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:33431
W20260807 20:16:02.613037 18259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:02.613305 18259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:02.613329 18259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:02.614492 18259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:02.614532 18259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:02.614557 18259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:16:02.616015 18259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:35053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=34901
--tserver_master_addrs=127.23.200.254:39057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:02.616252 18259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:02.616467 18259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:02.621475 18270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:02.621475 18267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:02.621469 18265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:02.621923 18259 server_base.cc:1034] running on GCE node
I20260807 20:16:02.622156 18259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:02.622400 18259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:02.623558 18259 hybrid_clock.cc:648] HybridClock initialized: now 1786133762623539 us; error 30 us; skew 500 ppm
Aug 07 20:16:02 dist-test-slave-2j7r krb5kdc[16785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133762, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:02.626163 18259 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:16:02.626623 18259 webserver.cc:460] Webserver started at http://127.23.200.195:34901/ using document root <none> and password file <none>
I20260807 20:16:02.626869 18259 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:02.626941 18259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:02.628331 18259 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:02.628916 18283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:02.629122 18259 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:02.629215 18259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "eb5b61d194c0458cb991dc85df1590c0"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "8894b0d59f0664516b3d3788d9ce5a76"
server_key_iv: "7b0e46a947fe53dae6203ae0610960db"
server_key_version: "encryptionkey@0"
I20260807 20:16:02.629555 18259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:02.655649 18259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:02.657614 18259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:02.657759 18259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:02.657994 18259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:02.658303 18259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:02.658342 18259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:02.658372 18259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:02.658389 18259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:02.664139 18259 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:35053
I20260807 20:16:02.664209 18464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:35053 every 8 connection(s)
I20260807 20:16:02.664543 18259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:16:02.665681 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 18259
Aug 07 20:16:02 dist-test-slave-2j7r krb5kdc[16785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133762, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:16:02.672987 18470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.666120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51700 (local address 127.23.200.254:39057)
0807 20:16:02.666273 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:02.666275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:02.666334 (+ 59us) server_negotiation.cc:407] Connection header received
0807 20:16:02.666407 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:02.666408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:02.666419 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:02.666495 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:02.667057 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.667729 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.668691 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.668951 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.669009 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.670197 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:02.670198 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:02.670199 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:02.670206 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:02.671636 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:02.672155 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:02.672156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:02.672156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:02.672221 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:02.672433 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:02.672434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:02.672434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:02.672572 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:02.672745 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:02.672783 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 20:16:02.672876 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":54,"threads_started":1}
I20260807 20:16:02.674381 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.665255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:39889 (local address 127.23.200.254:39057)
0807 20:16:02.665326 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:02.665328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:02.665544 (+ 216us) server_negotiation.cc:407] Connection header received
0807 20:16:02.666195 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:02.666198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:02.666210 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:02.666284 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:02.667378 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.667953 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.669185 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.669494 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.669555 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.671709 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:02.671710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:02.671710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:02.671717 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:02.673032 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:02.673484 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:02.673485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:02.673485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:02.673540 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:02.673785 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:02.673786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:02.673786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:02.673902 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:02.673954 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:02.674259 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 20:16:02.674320 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 20:16:02.674670 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.665246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:39889)
0807 20:16:02.665413 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:02.665455 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:02.666081 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:02.666377 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:02.666381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:02.667009 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:02.667284 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:02.667291 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.668044 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.668045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:02.669079 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:02.669112 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.669576 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.669577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:02.669686 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.670106 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:16:02.670115 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:02.671624 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:02.673128 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:02.673135 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:02.673137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:02.673430 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:02.673594 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:02.673595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:02.673595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:02.673719 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:02.673994 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:02.674001 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:02.674139 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 20:16:02.674241 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 20:16:02.674436 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260807 20:16:02.675087 18465 heartbeater.cc:347] Connected to a master server at 127.23.200.254:39057
I20260807 20:16:02.675189 18465 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:02.675372 18465 heartbeater.cc:511] Master 127.23.200.254:39057 requested a full tablet report, sending...
I20260807 20:16:02.675803 17729 ts_manager.cc:194] Registered new tserver with Master: eb5b61d194c0458cb991dc85df1590c0 (127.23.200.195:35053)
I20260807 20:16:02.676163 17729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:39889
I20260807 20:16:02.684554 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:16:02.687271 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.684956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51702 (local address 127.23.200.254:39057)
0807 20:16:02.685001 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:02.685002 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:02.685076 (+ 74us) server_negotiation.cc:407] Connection header received
0807 20:16:02.685185 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:02.685186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:02.685231 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:02.685307 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:16:02.685431 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.685918 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.686636 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.686839 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.686892 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.687037 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:16:02.687085 (+ 48us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 20:16:02.687204 (+ 119us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51702: token expired
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 20:16:02.687325 18034 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51702: token expired
W20260807 20:16:02.687484 17608 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.23.200.254:39057)
I20260807 20:16:02.692816 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.687725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51704 (local address 127.23.200.254:39057)
0807 20:16:02.687757 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:02.687759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:02.687767 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:16:02.687813 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:02.687813 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:02.687821 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:02.687861 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:02.688346 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.688775 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.689472 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.689676 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.689731 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.690762 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:02.690763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:02.690764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:02.690770 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:02.692034 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:02.692285 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:02.692286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:02.692286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:02.692331 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:02.692457 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:02.692457 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:02.692458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:02.692572 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:02.692643 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:02.692703 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 20:16:02.692758 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260807 20:16:02.693997 17729 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:16:02.695552 17729 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:16:02.697932 17914 tablet_service.cc:1467] Processing CreateTablet for tablet c2f4200258214d3e8a50e258c588abd9 (DEFAULT_TABLE table=test-table [id=560ab4f6acbc4cc29809baf7795916cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:02.698148 17914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2f4200258214d3e8a50e258c588abd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:02.699645 17852 tablet_bootstrap.cc:492] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb: Bootstrap starting.
I20260807 20:16:02.699930 17852 tablet_bootstrap.cc:654] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:02.700400 17852 tablet_bootstrap.cc:492] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb: No bootstrap required, opened a new log
I20260807 20:16:02.700448 17852 ts_tablet_manager.cc:1397] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260807 20:16:02.700557 17852 raft_consensus.cc:348] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:02.700623 17852 raft_consensus.cc:374] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:02.700662 17852 raft_consensus.cc:729] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27e9ff6e9b814a7b84940d772a026dbb, State: Initialized, Role: FOLLOWER
I20260807 20:16:02.700703 17852 consensus_queue.cc:260] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:02.700789 17852 ts_tablet_manager.cc:1428] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:02.701449 18038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.697337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:35053 (local address 127.0.0.1:43898)
0807 20:16:02.697388 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:02.697405 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:02.697444 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:02.697903 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:02.697904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:02.697905 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:02.697948 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:02.697949 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.699037 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.699038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:02.700090 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:02.700093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.701228 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.701229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:02.701286 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.701290 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:16:02.701318 (+ 28us) client_negotiation.cc:240] Negotiation successful
0807 20:16:02.701363 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":3}
I20260807 20:16:02.701537 18474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.697360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:41909 (local address 127.0.0.1:37938)
0807 20:16:02.697520 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:02.697529 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:02.697585 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:02.698001 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:02.698002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:02.698004 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:02.698048 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:02.698049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.699265 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.699266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:02.700299 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:02.700301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.701335 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.701336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:02.701385 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.701388 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:16:02.701438 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 20:16:02.701472 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":2,"thread_start_us":111,"threads_started":1}
I20260807 20:16:02.701627 18475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.697554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37938 (local address 127.23.200.194:41909)
0807 20:16:02.697745 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:02.697750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:02.697762 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:16:02.697804 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:02.697806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:02.697821 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:02.697900 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:02.698122 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.699172 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.700475 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.701184 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.701244 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.701304 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:02.701468 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 20:16:02.701544 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":73,"threads_started":1}
I20260807 20:16:02.701769 18478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.697606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43898 (local address 127.23.200.195:35053)
0807 20:16:02.697769 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:02.697773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:02.697780 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:16:02.697811 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:02.697813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:02.697826 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:02.697902 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:02.697991 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.698972 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.700181 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.701130 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.701556 (+ 426us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.701624 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:02.701651 (+ 27us) server_negotiation.cc:299] Negotiation successful
0807 20:16:02.701691 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260807 20:16:02.702272 18142 tablet_service.cc:1467] Processing CreateTablet for tablet c2f4200258214d3e8a50e258c588abd9 (DEFAULT_TABLE table=test-table [id=560ab4f6acbc4cc29809baf7795916cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:02.702570 18142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2f4200258214d3e8a50e258c588abd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:02.705305 18481 tablet_bootstrap.cc:492] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42: Bootstrap starting.
I20260807 20:16:02.705722 18357 tablet_service.cc:1467] Processing CreateTablet for tablet c2f4200258214d3e8a50e258c588abd9 (DEFAULT_TABLE table=test-table [id=560ab4f6acbc4cc29809baf7795916cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:02.705961 18481 tablet_bootstrap.cc:654] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:02.706023 18357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2f4200258214d3e8a50e258c588abd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:02.706492 18481 log.cc:824] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:02.707237 18481 tablet_bootstrap.cc:492] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42: No bootstrap required, opened a new log
I20260807 20:16:02.707298 18481 ts_tablet_manager.cc:1397] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:16:02.708778 18484 tablet_bootstrap.cc:492] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0: Bootstrap starting.
I20260807 20:16:02.708702 18481 raft_consensus.cc:348] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:02.708863 18481 raft_consensus.cc:374] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:02.708956 18481 raft_consensus.cc:729] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b61b5d275cd492aad3c527a4a928d42, State: Initialized, Role: FOLLOWER
I20260807 20:16:02.709122 18481 consensus_queue.cc:260] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:02.709401 18481 ts_tablet_manager.cc:1428] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:16:02.709432 18255 heartbeater.cc:503] Master 127.23.200.254:39057 was elected leader, sending a full tablet report...
I20260807 20:16:02.709604 18484 tablet_bootstrap.cc:654] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:02.710125 18484 log.cc:824] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:02.710819 18484 tablet_bootstrap.cc:492] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0: No bootstrap required, opened a new log
I20260807 20:16:02.710878 18484 ts_tablet_manager.cc:1397] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:16:02.712174 18484 raft_consensus.cc:348] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:02.712342 18484 raft_consensus.cc:374] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:02.712397 18484 raft_consensus.cc:729] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb5b61d194c0458cb991dc85df1590c0, State: Initialized, Role: FOLLOWER
I20260807 20:16:02.712512 18484 consensus_queue.cc:260] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:02.712776 18484 ts_tablet_manager.cc:1428] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:16:02.712822 18465 heartbeater.cc:503] Master 127.23.200.254:39057 was elected leader, sending a full tablet report...
I20260807 20:16:02.732460 18486 raft_consensus.cc:482] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:16:02.732605 18486 raft_consensus.cc:504] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:02.732983 18486 leader_election.cc:288] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb5b61d194c0458cb991dc85df1590c0 (127.23.200.195:35053), 27e9ff6e9b814a7b84940d772a026dbb (127.23.200.193:34185)
I20260807 20:16:02.735864 18478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.733222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:55089 (local address 127.23.200.195:35053)
0807 20:16:02.733275 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:02.733278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:02.733514 (+ 236us) server_negotiation.cc:407] Connection header received
0807 20:16:02.733583 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:02.733584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:02.733595 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:02.733646 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:02.733784 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.734229 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.735187 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.735559 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.735621 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.735629 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:02.735759 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 20:16:02.735796 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:16:02.735891 18257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.733202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:34185 (local address 127.23.200.194:45429)
0807 20:16:02.733284 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:02.733298 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:02.733398 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:02.733588 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:02.733589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:02.733590 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:02.733669 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:02.733670 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.734308 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.734309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:02.735103 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:02.735105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.735624 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.735625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:02.735706 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.735709 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:16:02.735756 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 20:16:02.735798 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":27}
I20260807 20:16:02.735877 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.733272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:45429 (local address 127.23.200.193:34185)
0807 20:16:02.733313 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:02.733315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:02.733376 (+ 61us) server_negotiation.cc:407] Connection header received
0807 20:16:02.733480 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:02.733481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:02.733491 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:02.733570 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:02.733737 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.734233 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.735193 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.735553 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.735615 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.735625 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:02.735775 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 20:16:02.735811 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:16:02.735915 18490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.733211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:35053 (local address 127.23.200.194:55089)
0807 20:16:02.733445 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:02.733456 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:02.733522 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:02.733655 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:02.733656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:02.733658 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:02.733721 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:02.733722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.734300 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.734301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:02.735104 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:02.735107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.735631 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.735632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:02.735699 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.735702 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:16:02.735746 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 20:16:02.735784 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":34,"thread_start_us":116,"threads_started":1}
I20260807 20:16:02.736259 18392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2f4200258214d3e8a50e258c588abd9" candidate_uuid: "7b61b5d275cd492aad3c527a4a928d42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5b61d194c0458cb991dc85df1590c0" is_pre_election: true
I20260807 20:16:02.736323 17948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2f4200258214d3e8a50e258c588abd9" candidate_uuid: "7b61b5d275cd492aad3c527a4a928d42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27e9ff6e9b814a7b84940d772a026dbb" is_pre_election: true
I20260807 20:16:02.736411 18392 raft_consensus.cc:2454] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b61b5d275cd492aad3c527a4a928d42 in term 0.
I20260807 20:16:02.736476 17948 raft_consensus.cc:2454] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b61b5d275cd492aad3c527a4a928d42 in term 0.
I20260807 20:16:02.736657 18066 leader_election.cc:302] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b61b5d275cd492aad3c527a4a928d42, eb5b61d194c0458cb991dc85df1590c0; no voters:
I20260807 20:16:02.736838 18486 raft_consensus.cc:2781] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:16:02.736891 18486 raft_consensus.cc:482] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:16:02.736915 18486 raft_consensus.cc:3037] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:02.737668 18486 raft_consensus.cc:504] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:02.737793 18486 leader_election.cc:288] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [CANDIDATE]: Term 1 election: Requested vote from peers eb5b61d194c0458cb991dc85df1590c0 (127.23.200.195:35053), 27e9ff6e9b814a7b84940d772a026dbb (127.23.200.193:34185)
I20260807 20:16:02.737958 17948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2f4200258214d3e8a50e258c588abd9" candidate_uuid: "7b61b5d275cd492aad3c527a4a928d42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27e9ff6e9b814a7b84940d772a026dbb"
I20260807 20:16:02.737994 18392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2f4200258214d3e8a50e258c588abd9" candidate_uuid: "7b61b5d275cd492aad3c527a4a928d42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb5b61d194c0458cb991dc85df1590c0"
I20260807 20:16:02.738060 18392 raft_consensus.cc:3037] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:02.738067 17948 raft_consensus.cc:3037] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:02.738765 18392 raft_consensus.cc:2454] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b61b5d275cd492aad3c527a4a928d42 in term 1.
I20260807 20:16:02.738766 17948 raft_consensus.cc:2454] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b61b5d275cd492aad3c527a4a928d42 in term 1.
I20260807 20:16:02.738936 18059 leader_election.cc:302] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27e9ff6e9b814a7b84940d772a026dbb, 7b61b5d275cd492aad3c527a4a928d42; no voters:
I20260807 20:16:02.739044 18486 raft_consensus.cc:2781] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:02.739199 18486 raft_consensus.cc:686] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [term 1 LEADER]: Becoming Leader. State: Replica: 7b61b5d275cd492aad3c527a4a928d42, State: Running, Role: LEADER
I20260807 20:16:02.739280 18486 consensus_queue.cc:237] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:02.740113 17729 catalog_manager.cc:5495] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b61b5d275cd492aad3c527a4a928d42 (127.23.200.194). New cstate: current_term: 1 leader_uuid: "7b61b5d275cd492aad3c527a4a928d42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } health_report { overall_health: UNKNOWN } } }
I20260807 20:16:02.750615 18475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.748401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37946 (local address 127.23.200.194:41909)
0807 20:16:02.748460 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:02.748461 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:02.748470 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:16:02.748533 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:02.748534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:02.748551 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:02.748609 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:16:02.748710 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.749223 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.749931 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:02.750124 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:02.750178 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:02.750317 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:16:02.750418 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:16:02.750489 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:02.750509 (+ 20us) server_negotiation.cc:299] Negotiation successful
0807 20:16:02.750528 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:16:02.752254 18392 raft_consensus.cc:1261] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0 [term 1 FOLLOWER]: Refusing update from remote peer 7b61b5d275cd492aad3c527a4a928d42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:16:02.752264 17948 raft_consensus.cc:1261] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Refusing update from remote peer 7b61b5d275cd492aad3c527a4a928d42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:16:02.752621 18491 consensus_queue.cc:1035] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:16:02.752738 18486 consensus_queue.cc:1035] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:16:02.755559 18500 mvcc.cc:205] Tried to move back new op lower bound from 7316003892230770688 to 7316003892180824064. Current Snapshot: MvccSnapshot[applied={T|T < 7316003892230770688 or (T in {7316003892230770688})}]
I20260807 20:16:02.757562 17729 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:16:02.757647 17729 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:16:02.758279 17729 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Sending DeleteTablet for 3 replicas of tablet c2f4200258214d3e8a50e258c588abd9
I20260807 20:16:02.758587 17914 tablet_service.cc:1514] Processing DeleteTablet for tablet c2f4200258214d3e8a50e258c588abd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:02 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:56558
I20260807 20:16:02.758594 18357 tablet_service.cc:1514] Processing DeleteTablet for tablet c2f4200258214d3e8a50e258c588abd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:02 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:43898
I20260807 20:16:02.758708 18142 tablet_service.cc:1514] Processing DeleteTablet for tablet c2f4200258214d3e8a50e258c588abd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:02 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:37938
I20260807 20:16:02.758876 18511 tablet_replica.cc:330] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0: stopping tablet replica
I20260807 20:16:02.758908 18040 tablet_replica.cc:330] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb: stopping tablet replica
I20260807 20:16:02.758944 18512 tablet_replica.cc:330] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42: stopping tablet replica
I20260807 20:16:02.758963 18040 raft_consensus.cc:2229] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:16:02.759009 18040 raft_consensus.cc:2258] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:02.759050 18512 raft_consensus.cc:2229] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:16:02.759171 18512 raft_consensus.cc:2258] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:02.759282 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 17782
I20260807 20:16:02.759428 18040 ts_tablet_manager.cc:1905] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:16:02.763773 18511 raft_consensus.cc:2229] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:16:02.763957 18511 raft_consensus.cc:2258] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:02.770201 18511 ts_tablet_manager.cc:1905] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:16:02.772004 18511 ts_tablet_manager.cc:1918] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:16:02.772078 18511 log.cc:1196] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/c2f4200258214d3e8a50e258c588abd9
I20260807 20:16:02.772320 18511 ts_tablet_manager.cc:1939] T c2f4200258214d3e8a50e258c588abd9 P eb5b61d194c0458cb991dc85df1590c0: Deleting consensus metadata
I20260807 20:16:02.772825 17704 catalog_manager.cc:4841] TS eb5b61d194c0458cb991dc85df1590c0 (127.23.200.195:35053): tablet c2f4200258214d3e8a50e258c588abd9 (table test-table [id=560ab4f6acbc4cc29809baf7795916cb]) successfully deleted
I20260807 20:16:02.779917 18512 ts_tablet_manager.cc:1905] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:16:02.781502 18512 ts_tablet_manager.cc:1918] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:16:02.781584 18512 log.cc:1196] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/c2f4200258214d3e8a50e258c588abd9
I20260807 20:16:02.781823 18512 ts_tablet_manager.cc:1939] T c2f4200258214d3e8a50e258c588abd9 P 7b61b5d275cd492aad3c527a4a928d42: Deleting consensus metadata
I20260807 20:16:02.782272 17700 catalog_manager.cc:4841] TS 7b61b5d275cd492aad3c527a4a928d42 (127.23.200.194:41909): tablet c2f4200258214d3e8a50e258c588abd9 (table test-table [id=560ab4f6acbc4cc29809baf7795916cb]) successfully deleted
W20260807 20:16:02.785234 17700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:34185 (error 108)
I20260807 20:16:02.785521 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 18036
I20260807 20:16:02.785802 18474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:02.785681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:34185 (local address 127.0.0.1:56568)
0807 20:16:02.785734 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:02.785767 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:34185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 20:16:02.785910 17700 catalog_manager.cc:4584] TS 27e9ff6e9b814a7b84940d772a026dbb (127.23.200.193:34185): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c2f4200258214d3e8a50e258c588abd9: Network error: Client connection negotiation failed: client connection to 127.23.200.193:34185: connect: Connection refused (error 111)
I20260807 20:16:02.797101 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 18259
I20260807 20:16:02.813424 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 17669
I20260807 20:16:22.824497 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:39057
--webserver_interface=127.23.200.254
--webserver_port=35451
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:39057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:22.913497 18514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:22.913722 18514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:22.913746 18514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:22.914927 18514 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 20:16:22.914979 18514 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 20:16:22.914994 18514 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 20:16:22.915009 18514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:16:22.915038 18514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:22.915066 18514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:22.915092 18514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:16:22.915120 18514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:16:22.916672 18514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:39057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:39057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=35451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:22.916929 18514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:22.917188 18514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:16:22.922694 18514 server_base.cc:1034] running on GCE node
W20260807 20:16:22.922647 18526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:22.922639 18528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:22.922657 18524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:22.923184 18514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:22.923480 18514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:22.924635 18514 hybrid_clock.cc:648] HybridClock initialized: now 1786133782924623 us; error 31 us; skew 500 ppm
Aug 07 20:16:22 dist-test-slave-2j7r krb5kdc[16785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133782, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:22.927219 18514 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:16:22.927677 18514 webserver.cc:460] Webserver started at http://127.23.200.254:35451/ using document root <none> and password file <none>
I20260807 20:16:22.927907 18514 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:22.927964 18514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:22.929317 18514 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:22.930049 18541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:22.930210 18514 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:22.930298 18514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "1b155e378287454db55e6d550036c7c5"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "c108ce4cad241ba081286831d85fa702"
server_key_iv: "65c9e786e21516db4bd26a3024e52f42"
server_key_version: "encryptionkey@0"
I20260807 20:16:22.930644 18514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:22.939539 18514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:22.941397 18514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:22.941533 18514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:22.945498 18514 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:39057
I20260807 20:16:22.945541 18623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:39057 every 8 connection(s)
I20260807 20:16:22.945904 18514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:16:22.947760 18625 sys_catalog.cc:263] Verifying existing consensus state
I20260807 20:16:22.948428 18625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Bootstrap starting.
I20260807 20:16:22.950723 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 18514
I20260807 20:16:22.951004 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:34185
--local_ip_for_outbound_sockets=127.23.200.193
--tserver_master_addrs=127.23.200.254:39057
--webserver_port=39743
--webserver_interface=127.23.200.193
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:16:22.956910 18625 log.cc:824] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:22.959092 18625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:16:22.959372 18625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Bootstrap complete.
I20260807 20:16:22.961500 18625 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } }
I20260807 20:16:22.961726 18625 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b155e378287454db55e6d550036c7c5, State: Initialized, Role: FOLLOWER
I20260807 20:16:22.961886 18625 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } }
I20260807 20:16:22.961947 18625 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:16:22.961994 18625 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:16:22.962041 18625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 2 FOLLOWER]: Advancing to term 3
I20260807 20:16:22.963121 18625 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } }
I20260807 20:16:22.963227 18625 leader_election.cc:302] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b155e378287454db55e6d550036c7c5; no voters:
I20260807 20:16:22.963421 18625 leader_election.cc:288] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 20:16:22.963474 18633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 20:16:22.963589 18633 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [term 3 LEADER]: Becoming Leader. State: Replica: 1b155e378287454db55e6d550036c7c5, State: Running, Role: LEADER
I20260807 20:16:22.963681 18633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } }
I20260807 20:16:22.963716 18625 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:16:22.963976 18634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1b155e378287454db55e6d550036c7c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } } }
I20260807 20:16:22.964067 18634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:22.964247 18635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b155e378287454db55e6d550036c7c5. Latest consensus state: current_term: 3 leader_uuid: "1b155e378287454db55e6d550036c7c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b155e378287454db55e6d550036c7c5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39057 } } }
I20260807 20:16:22.964300 18635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:22.964367 18638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:16:22.965487 18638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:16:22.965655 18638 catalog_manager.cc:1259] Loaded cluster ID: d2ddc1a3009848e58e309703d062b19c
I20260807 20:16:22.965936 18638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:16:22.967209 18638 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:16:22.967377 18638 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Loaded TSK: 0
I20260807 20:16:22.967418 18638 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1b155e378287454db55e6d550036c7c5: Loaded TSK: 1
F20260807 20:16:22.967655 18638 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f11e7b959ad google::LogMessage::Fail() at ??:0
@ 0x7f11e7b9888a google::LogMessage::SendToLog() at ??:0
@ 0x7f11e7b954de google::LogMessage::Flush() at ??:0
@ 0x7f11e7b96a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f11ea302104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f11ea2ffd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f11efaf31cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f11efaf348e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f11efaf3594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f11efab9c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f11efac3ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f11e9a630d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f11e9a58954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f11ea8bc6db start_thread at ??:0
@ 0x7f11e669361f clone at ??:0
W20260807 20:16:23.071269 18628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:23.071548 18628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:23.071584 18628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:23.073386 18628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:23.073454 18628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:23.073495 18628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:16:23.075748 18628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:34185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=39743
--tserver_master_addrs=127.23.200.254:39057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:23.076050 18628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:23.076321 18628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:23.081852 18702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:23.081862 18701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:23.081862 18704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:23.082443 18628 server_base.cc:1034] running on GCE node
I20260807 20:16:23.082703 18628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:23.082998 18628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:23.084175 18628 hybrid_clock.cc:648] HybridClock initialized: now 1786133783084160 us; error 28 us; skew 500 ppm
Aug 07 20:16:23 dist-test-slave-2j7r krb5kdc[16785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133783, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:23.087352 18628 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:16:23.087901 18628 webserver.cc:460] Webserver started at http://127.23.200.193:39743/ using document root <none> and password file <none>
I20260807 20:16:23.088169 18628 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:23.088224 18628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:23.089957 18628 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:16:23.090854 18717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:23.091058 18628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:23.091157 18628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "27e9ff6e9b814a7b84940d772a026dbb"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "859365d848584977431a9ced20484524"
server_key_iv: "9c89283a98a997f747741335d644e9a5"
server_key_version: "encryptionkey@0"
I20260807 20:16:23.091588 18628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:23.103461 18628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:23.105715 18628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:23.105880 18628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:23.106145 18628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:23.106679 18728 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 20:16:23.108150 18628 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 20:16:23.108212 18628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 20:16:23.108243 18628 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 20:16:23.109191 18628 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 20:16:23.109243 18628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:23.109292 18728 tablet_bootstrap.cc:492] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb: Bootstrap starting.
I20260807 20:16:23.110531 18728 log.cc:824] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:23.113404 18728 tablet_bootstrap.cc:492] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:16:23.113706 18728 tablet_bootstrap.cc:492] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb: Bootstrap complete.
I20260807 20:16:23.113866 18728 ts_tablet_manager.cc:1397] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 20:16:23.115641 18728 raft_consensus.cc:348] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:23.115975 18728 raft_consensus.cc:729] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27e9ff6e9b814a7b84940d772a026dbb, State: Initialized, Role: FOLLOWER
I20260807 20:16:23.116176 18728 consensus_queue.cc:260] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:23.116443 18728 ts_tablet_manager.cc:1428] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:16:23.116730 18628 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:34185
I20260807 20:16:23.116761 18917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:34185 every 8 connection(s)
I20260807 20:16:23.117153 18628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:16:23.117734 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 18628
I20260807 20:16:23.118037 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:41909
--local_ip_for_outbound_sockets=127.23.200.194
--tserver_master_addrs=127.23.200.254:39057
--webserver_port=43655
--webserver_interface=127.23.200.194
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:23.191887 18920 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:16:23.117885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:37345)
0807 20:16:23.118052 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:23.118094 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:23.118248 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:23.191696 (+ 73448us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
W20260807 20:16:23.192173 18918 heartbeater.cc:644] Failed to heartbeat to 127.23.200.254:39057 (0 consecutive failures): Network error: Failed to ping master at 127.23.200.254:39057: Client connection negotiation failed: client connection to 127.23.200.254:39057: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 20:16:23.192523 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:23.192405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:48457)
0807 20:16:23.192458 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:23.192500 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 20:16:23.192893 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:23.192792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:55571)
0807 20:16:23.192845 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:23.192874 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 20:16:23.193012 18918 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 20:16:23.207562 18921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:23.207767 18921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:23.207790 18921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:23.208943 18921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:23.208988 18921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:23.209013 18921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:16:23.210521 18921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:41909
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=43655
--tserver_master_addrs=127.23.200.254:39057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:23.210752 18921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:23.210978 18921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:23.216104 18934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:23.216112 18930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:23.216112 18931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:23.216362 18921 server_base.cc:1034] running on GCE node
I20260807 20:16:23.216564 18921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:23.216786 18921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:23.217923 18921 hybrid_clock.cc:648] HybridClock initialized: now 1786133783217911 us; error 27 us; skew 500 ppm
Aug 07 20:16:23 dist-test-slave-2j7r krb5kdc[16785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133783, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:23.220417 18921 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:16:23.220837 18921 webserver.cc:460] Webserver started at http://127.23.200.194:43655/ using document root <none> and password file <none>
I20260807 20:16:23.221052 18921 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:23.221129 18921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:23.222476 18921 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:16:23.223156 18943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:23.223344 18921 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:16:23.223452 18921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "7b61b5d275cd492aad3c527a4a928d42"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "0a7116fcef202f3df9bb6496fb00a531"
server_key_iv: "eab3bfe1c4fa3f92ac5e0701f1f07699"
server_key_version: "encryptionkey@0"
I20260807 20:16:23.223819 18921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:23.233264 18921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:23.235075 18921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:23.235230 18921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:23.235446 18921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:23.235744 18921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:23.235780 18921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:23.235813 18921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:23.235836 18921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:23.242133 18921 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:41909
I20260807 20:16:23.242194 19115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:41909 every 8 connection(s)
I20260807 20:16:23.242544 18921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:16:23.243621 19119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:23.243211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:50067)
0807 20:16:23.243402 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:23.243479 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":79,"threads_started":1}
W20260807 20:16:23.244030 19117 heartbeater.cc:644] Failed to heartbeat to 127.23.200.254:39057 (0 consecutive failures): Network error: Failed to ping master at 127.23.200.254:39057: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
I20260807 20:16:23.244228 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 18921
I20260807 20:16:23.244319 19119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:23.244204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:33977)
0807 20:16:23.244256 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:23.244291 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 20:16:23.244549 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:35053
--local_ip_for_outbound_sockets=127.23.200.195
--tserver_master_addrs=127.23.200.254:39057
--webserver_port=34901
--webserver_interface=127.23.200.195
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:16:23.244642 19119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:23.244554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:47915)
0807 20:16:23.244593 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:23.244621 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 20:16:23.244791 19117 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 20:16:23.328503 19120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:23.328712 19120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:23.328743 19120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:23.329972 19120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:23.330020 19120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:23.330057 19120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:16:23.331542 19120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:35053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=34901
--tserver_master_addrs=127.23.200.254:39057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:23.331789 19120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:23.332012 19120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:23.337169 19128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:23.337203 19132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:23.337169 19129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:23.337401 19120 server_base.cc:1034] running on GCE node
I20260807 20:16:23.337606 19120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:23.337821 19120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:23.338961 19120 hybrid_clock.cc:648] HybridClock initialized: now 1786133783338945 us; error 32 us; skew 500 ppm
Aug 07 20:16:23 dist-test-slave-2j7r krb5kdc[16785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133783, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:23.341393 19120 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:16:23.341861 19120 webserver.cc:460] Webserver started at http://127.23.200.195:34901/ using document root <none> and password file <none>
I20260807 20:16:23.342095 19120 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:23.342145 19120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:23.343536 19120 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:23.344106 19142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:23.344249 19120 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:23.344333 19120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "eb5b61d194c0458cb991dc85df1590c0"
format_stamp: "Formatted at 2026-08-07 20:15:41 on dist-test-slave-2j7r"
server_key: "8894b0d59f0664516b3d3788d9ce5a76"
server_key_iv: "7b0e46a947fe53dae6203ae0610960db"
server_key_version: "encryptionkey@0"
I20260807 20:16:23.344666 19120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:23.356011 19120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:23.357744 19120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:23.357882 19120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:23.358084 19120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:23.358376 19120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:23.358409 19120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:23.358438 19120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:23.358460 19120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:23.364127 19120 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:35053
I20260807 20:16:23.364171 19323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:35053 every 8 connection(s)
I20260807 20:16:23.364514 19120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:16:23.365556 19328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:23.365183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:48431)
0807 20:16:23.365338 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:23.365414 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
W20260807 20:16:23.365916 19325 heartbeater.cc:644] Failed to heartbeat to 127.23.200.254:39057 (0 consecutive failures): Network error: Failed to ping master at 127.23.200.254:39057: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
I20260807 20:16:23.366206 19328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:23.366119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:56187)
0807 20:16:23.366163 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:23.366189 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 20:16:23.366489 19328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:23.366433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:48929)
0807 20:16:23.366457 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:23.366477 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260807 20:16:23.366581 19325 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 20:16:23.370008 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 19120
I20260807 20:16:24.193637 19332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:24.193414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:58077)
0807 20:16:24.193538 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:24.193573 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260807 20:16:24.245364 19334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:24.245128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:57413)
0807 20:16:24.245277 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:24.245310 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260807 20:16:24.367143 19336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:24.366917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:55011)
0807 20:16:24.367052 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:24.367090 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260807 20:16:24.620978 19339 raft_consensus.cc:482] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:16:24.621151 19339 raft_consensus.cc:504] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:24.621486 19339 leader_election.cc:288] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb5b61d194c0458cb991dc85df1590c0 (127.23.200.195:35053), 7b61b5d275cd492aad3c527a4a928d42 (127.23.200.194:41909)
Aug 07 20:16:24 dist-test-slave-2j7r krb5kdc[16785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133783, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.194@KRBTEST.COM
Aug 07 20:16:24 dist-test-slave-2j7r krb5kdc[16785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133783, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.195@KRBTEST.COM
I20260807 20:16:24.634709 19332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:24.621657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:41909 (local address 127.23.200.193:41887)
0807 20:16:24.621713 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:24.621726 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:24.621829 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:24.622670 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:24.622673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:24.623049 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:24.623490 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:24.623493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:24.624566 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:24.624567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:24.625184 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:24.625187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:24.625841 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:24.625843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:24.625984 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:24.629455 (+ 3471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:16:24.629465 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:24.631073 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:24.633159 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:24.633167 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:24.633170 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:24.633479 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:24.633726 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:24.633727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:24.633727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:24.633832 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:24.634205 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:24.634212 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:24.634324 (+ 112us) client_negotiation.cc:769] Sending connection context
0807 20:16:24.634406 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 20:16:24.634588 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":2}
I20260807 20:16:24.634747 19347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:24.621820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:43597 (local address 127.23.200.195:35053)
0807 20:16:24.621981 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:24.621990 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:24.622017 (+ 27us) server_negotiation.cc:407] Connection header received
0807 20:16:24.622422 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:24.622424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:24.622475 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:24.622603 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:24.623385 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:24.624263 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:24.625282 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:24.625749 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:24.625852 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:24.631495 (+ 5643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:24.631500 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:24.631500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:24.631512 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:24.633151 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:24.633555 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:24.633559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:24.633559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:24.633655 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:24.633888 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:24.633889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:24.633889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:24.634138 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:24.634211 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:24.634528 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 20:16:24.634655 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":49,"threads_started":1}
I20260807 20:16:24.634788 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:24.621657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:35053 (local address 127.23.200.193:43597)
0807 20:16:24.621807 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:24.621817 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:24.621877 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:24.622670 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:24.622673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:24.623058 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:24.623266 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:24.623271 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:24.624359 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:24.624360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:24.625182 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:24.625185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:24.625835 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:24.625836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:24.625970 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:24.629462 (+ 3492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:16:24.629468 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:24.631383 (+ 1915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:24.633231 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:24.633232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:24.633233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:24.633477 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:24.633718 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:24.633719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:24.633719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:24.633827 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:24.634205 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:24.634212 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:24.634323 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 20:16:24.634455 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 20:16:24.634659 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":152,"thread_start_us":49,"threads_started":1}
I20260807 20:16:24.634733 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:24.621783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:41887 (local address 127.23.200.194:41909)
0807 20:16:24.621940 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:24.621953 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:24.621985 (+ 32us) server_negotiation.cc:407] Connection header received
0807 20:16:24.622448 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:24.622451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:24.622491 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:24.622615 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:24.623611 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:24.624467 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:24.625270 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:24.625750 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:24.625857 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:24.631301 (+ 5444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:24.631311 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:24.631321 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:24.631337 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:24.633055 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:24.633562 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:24.633566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:24.633566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:24.633660 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:24.633894 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:24.633895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:24.633895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:24.634138 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:24.634210 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:24.634482 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 20:16:24.634617 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
W20260807 20:16:24.635619 18722 leader_election.cc:341] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb5b61d194c0458cb991dc85df1590c0 (127.23.200.195:35053): Not found: Tablet not found: c2f4200258214d3e8a50e258c588abd9
W20260807 20:16:24.635737 18719 leader_election.cc:341] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b61b5d275cd492aad3c527a4a928d42 (127.23.200.194:41909): Not found: Tablet not found: c2f4200258214d3e8a50e258c588abd9
I20260807 20:16:24.635803 18719 leader_election.cc:302] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27e9ff6e9b814a7b84940d772a026dbb; no voters: 7b61b5d275cd492aad3c527a4a928d42, eb5b61d194c0458cb991dc85df1590c0
I20260807 20:16:24.635915 19339 raft_consensus.cc:2726] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 20:16:25.194334 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:25.194118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:41419)
0807 20:16:25.194248 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:25.194283 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260807 20:16:25.246105 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:25.245869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:47519)
0807 20:16:25.246014 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:25.246061 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
I20260807 20:16:25.367947 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:25.367701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:40907)
0807 20:16:25.367854 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:25.367901 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
I20260807 20:16:26.195138 19353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:26.194849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:36579)
0807 20:16:26.195017 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:26.195068 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":87,"threads_started":1}
I20260807 20:16:26.246881 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:26.246646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:48259)
0807 20:16:26.246789 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:26.246837 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260807 20:16:26.368768 19355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:26.368504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:49915)
0807 20:16:26.368651 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:26.368701 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260807 20:16:26.851670 19356 raft_consensus.cc:482] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:16:26.851773 19356 raft_consensus.cc:504] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:26.851926 19356 leader_election.cc:288] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb5b61d194c0458cb991dc85df1590c0 (127.23.200.195:35053), 7b61b5d275cd492aad3c527a4a928d42 (127.23.200.194:41909)
W20260807 20:16:26.852357 18722 leader_election.cc:341] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb5b61d194c0458cb991dc85df1590c0 (127.23.200.195:35053): Not found: Tablet not found: c2f4200258214d3e8a50e258c588abd9
W20260807 20:16:26.852468 18719 leader_election.cc:341] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b61b5d275cd492aad3c527a4a928d42 (127.23.200.194:41909): Not found: Tablet not found: c2f4200258214d3e8a50e258c588abd9
I20260807 20:16:26.852501 18719 leader_election.cc:302] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27e9ff6e9b814a7b84940d772a026dbb; no voters: 7b61b5d275cd492aad3c527a4a928d42, eb5b61d194c0458cb991dc85df1590c0
I20260807 20:16:26.852596 19356 raft_consensus.cc:2726] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 20:16:27.195768 19357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:27.195546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:41299)
0807 20:16:27.195662 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:27.195699 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":61,"threads_started":1}
I20260807 20:16:27.247674 19358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:27.247409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:33203)
0807 20:16:27.247563 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:27.247609 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
I20260807 20:16:27.369572 19359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:27.369315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:54083)
0807 20:16:27.369470 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:27.369518 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":67,"threads_started":1}
I20260807 20:16:28.196475 19361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:28.196213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:36583)
0807 20:16:28.196376 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:28.196427 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":128,"threads_started":1}
I20260807 20:16:28.248368 19362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:28.248135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:37045)
0807 20:16:28.248275 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:28.248318 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260807 20:16:28.370360 19363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:28.370128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:40669)
0807 20:16:28.370264 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:28.370311 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
I20260807 20:16:29.109476 19364 raft_consensus.cc:482] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:16:29.109589 19364 raft_consensus.cc:504] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:29.109797 19364 leader_election.cc:288] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb5b61d194c0458cb991dc85df1590c0 (127.23.200.195:35053), 7b61b5d275cd492aad3c527a4a928d42 (127.23.200.194:41909)
W20260807 20:16:29.110232 18719 leader_election.cc:341] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b61b5d275cd492aad3c527a4a928d42 (127.23.200.194:41909): Not found: Tablet not found: c2f4200258214d3e8a50e258c588abd9
W20260807 20:16:29.110318 18722 leader_election.cc:341] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb5b61d194c0458cb991dc85df1590c0 (127.23.200.195:35053): Not found: Tablet not found: c2f4200258214d3e8a50e258c588abd9
I20260807 20:16:29.110368 18722 leader_election.cc:302] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27e9ff6e9b814a7b84940d772a026dbb; no voters: 7b61b5d275cd492aad3c527a4a928d42, eb5b61d194c0458cb991dc85df1590c0
I20260807 20:16:29.110443 19364 raft_consensus.cc:2726] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 20:16:29.197211 19365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:29.196931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:45721)
0807 20:16:29.197115 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:29.197161 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
I20260807 20:16:29.249123 19367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:29.248822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:44037)
0807 20:16:29.248990 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:29.249039 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":70,"threads_started":1}
I20260807 20:16:29.371112 19368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:29.370880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:49153)
0807 20:16:29.371037 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:29.371069 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260807 20:16:30.197927 19370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:30.197674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:37087)
0807 20:16:30.197827 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:30.197872 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
I20260807 20:16:30.249902 19371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:30.249657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:43439)
0807 20:16:30.249798 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:30.249854 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":64,"threads_started":1}
I20260807 20:16:30.371922 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:30.371614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:36951)
0807 20:16:30.371815 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:30.371863 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":94,"threads_started":1}
I20260807 20:16:31.198654 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:31.198397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:60609)
0807 20:16:31.198549 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:31.198603 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":68,"threads_started":1}
I20260807 20:16:31.250583 19375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:31.250373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:56783)
0807 20:16:31.250495 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:31.250537 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260807 20:16:31.372671 19379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:31.372430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:52453)
0807 20:16:31.372582 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:31.372623 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":64,"threads_started":1}
I20260807 20:16:32.199472 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:32.199182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:35989)
0807 20:16:32.199367 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:32.199420 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260807 20:16:32.251267 19382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:32.251039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:34965)
0807 20:16:32.251171 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:32.251211 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":60,"threads_started":1}
I20260807 20:16:32.373359 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:32.373129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:60261)
0807 20:16:32.373268 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:32.373310 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":61,"threads_started":1}
I20260807 20:16:32.914053 19384 raft_consensus.cc:482] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:16:32.914166 19384 raft_consensus.cc:504] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb5b61d194c0458cb991dc85df1590c0" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35053 } } peers { permanent_uuid: "7b61b5d275cd492aad3c527a4a928d42" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 41909 } } peers { permanent_uuid: "27e9ff6e9b814a7b84940d772a026dbb" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34185 } }
I20260807 20:16:32.914361 19384 leader_election.cc:288] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb5b61d194c0458cb991dc85df1590c0 (127.23.200.195:35053), 7b61b5d275cd492aad3c527a4a928d42 (127.23.200.194:41909)
W20260807 20:16:32.914888 18722 leader_election.cc:341] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb5b61d194c0458cb991dc85df1590c0 (127.23.200.195:35053): Not found: Tablet not found: c2f4200258214d3e8a50e258c588abd9
W20260807 20:16:32.914983 18719 leader_election.cc:341] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b61b5d275cd492aad3c527a4a928d42 (127.23.200.194:41909): Not found: Tablet not found: c2f4200258214d3e8a50e258c588abd9
I20260807 20:16:32.915030 18719 leader_election.cc:302] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27e9ff6e9b814a7b84940d772a026dbb; no voters: 7b61b5d275cd492aad3c527a4a928d42, eb5b61d194c0458cb991dc85df1590c0
I20260807 20:16:32.915118 19384 raft_consensus.cc:2726] T c2f4200258214d3e8a50e258c588abd9 P 27e9ff6e9b814a7b84940d772a026dbb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 20:16:33.200248 19386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:33.200004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:54481)
0807 20:16:33.200158 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:33.200199 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260807 20:16:33.251981 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:33.251750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:38297)
0807 20:16:33.251891 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:33.251933 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":63,"threads_started":1}
I20260807 20:16:33.374002 19389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:33.373770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:43051)
0807 20:16:33.373919 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:33.373957 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260807 20:16:34.201016 19391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:34.200747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:50103)
0807 20:16:34.200914 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:34.200965 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":80,"threads_started":1}
I20260807 20:16:34.252799 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:34.252519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:43415)
0807 20:16:34.252679 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:34.252723 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":79,"threads_started":1}
I20260807 20:16:34.374766 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:34.374516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:41925)
0807 20:16:34.374673 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:34.374716 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":64,"threads_started":1}
I20260807 20:16:35.201797 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:35.201563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:46447)
0807 20:16:35.201690 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:35.201746 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260807 20:16:35.253604 19399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:35.253357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:44083)
0807 20:16:35.253509 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:35.253553 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":63,"threads_started":1}
I20260807 20:16:35.375550 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:35.375298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:54889)
0807 20:16:35.375455 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:35.375500 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":68,"threads_started":1}
I20260807 20:16:36.202556 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:36.202328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:46729)
0807 20:16:36.202466 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:36.202508 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260807 20:16:36.254402 19406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:36.254167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:33925)
0807 20:16:36.254307 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:36.254352 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260807 20:16:36.376303 19407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:36.376050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:57707)
0807 20:16:36.376210 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:36.376255 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":66,"threads_started":1}
I20260807 20:16:37.203240 19408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:37.203003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:51549)
0807 20:16:37.203141 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:37.203188 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260807 20:16:37.255167 19410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:37.254917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:49361)
0807 20:16:37.255072 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:37.255118 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":76,"threads_started":1}
I20260807 20:16:37.377228 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:37.376972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:60091)
0807 20:16:37.377132 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:37.377179 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":69,"threads_started":1}
I20260807 20:16:38.204052 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:38.203776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.193:48191)
0807 20:16:38.203962 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:38.203999 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260807 20:16:38.255916 19413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:38.255677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.194:43627)
0807 20:16:38.255835 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:38.255867 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":112,"threads_started":1}
I20260807 20:16:38.370733 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 18628
I20260807 20:16:38.379329 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:38.379048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:39057 (local address 127.23.200.195:34865)
0807 20:16:38.379246 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:38.379282 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.254:39057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":52,"threads_started":1}
I20260807 20:16:38.383773 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 18921
I20260807 20:16:38.407341 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 19120
2026-08-07T20:16:38Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58561 ms)
[----------] 30 tests from SecurityITest (236342 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:16:38 dist-test-slave-2j7r krb5kdc[19422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:16:38 dist-test-slave-2j7r krb5kdc[19422](info): set up 2 sockets
Aug 07 20:16:38 dist-test-slave-2j7r krb5kdc[19422](info): commencing operation
krb5kdc: starting...
W20260807 20:16:39.593905 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:16:39 dist-test-slave-2j7r krb5kdc[19422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:16:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:16:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:16:39.721300 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:38169
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:38169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:39.805593 19446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:39.805815 19446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:39.805842 19446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:39.807055 19446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:16:39.807097 19446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:39.807111 19446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:39.807123 19446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:16:39.807145 19446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:16:39.808655 19446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:38169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:38169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:39.808893 19446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:39.809134 19446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:39.814230 19459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:39.814262 19454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:39.814296 19446 server_base.cc:1034] running on GCE node
W20260807 20:16:39.814246 19456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:39.814664 19446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:39.814939 19446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:39.816097 19446 hybrid_clock.cc:648] HybridClock initialized: now 1786133799816080 us; error 28 us; skew 500 ppm
Aug 07 20:16:39 dist-test-slave-2j7r krb5kdc[19422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133799, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:39.818689 19446 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:16:39.819072 19446 webserver.cc:460] Webserver started at http://127.23.200.254:33479/ using document root <none> and password file <none>
I20260807 20:16:39.819278 19446 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:39.819319 19446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:39.819415 19446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:39.820426 19446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "1cffe81cc5da4154b0c721b07d5d82d2"
format_stamp: "Formatted at 2026-08-07 20:16:39 on dist-test-slave-2j7r"
server_key: "f5eef6d0c7d88ae56ba29a51d542df47"
server_key_iv: "38b9adce828c45def622af20490645b1"
server_key_version: "encryptionkey@0"
I20260807 20:16:39.820727 19446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "1cffe81cc5da4154b0c721b07d5d82d2"
format_stamp: "Formatted at 2026-08-07 20:16:39 on dist-test-slave-2j7r"
server_key: "f5eef6d0c7d88ae56ba29a51d542df47"
server_key_iv: "38b9adce828c45def622af20490645b1"
server_key_version: "encryptionkey@0"
I20260807 20:16:39.822009 19446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:16:39.822721 19469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:39.822898 19446 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:16:39.822988 19446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "1cffe81cc5da4154b0c721b07d5d82d2"
format_stamp: "Formatted at 2026-08-07 20:16:39 on dist-test-slave-2j7r"
server_key: "f5eef6d0c7d88ae56ba29a51d542df47"
server_key_iv: "38b9adce828c45def622af20490645b1"
server_key_version: "encryptionkey@0"
I20260807 20:16:39.823043 19446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:39.830896 19446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:39.832845 19446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:39.832967 19446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:39.836899 19446 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:38169
I20260807 20:16:39.836944 19559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:38169 every 8 connection(s)
I20260807 20:16:39.837328 19446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:16:39.838270 19561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:39.840669 19561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2: Bootstrap starting.
I20260807 20:16:39.841336 19561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:39.841621 19561 log.cc:824] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:39.842326 19561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2: No bootstrap required, opened a new log
I20260807 20:16:39.843957 19561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cffe81cc5da4154b0c721b07d5d82d2" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38169 } }
I20260807 20:16:39.844103 19561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:39.844128 19561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cffe81cc5da4154b0c721b07d5d82d2, State: Initialized, Role: FOLLOWER
I20260807 20:16:39.844267 19561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cffe81cc5da4154b0c721b07d5d82d2" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38169 } }
I20260807 20:16:39.844344 19561 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:16:39.844369 19561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:16:39.844408 19561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:39.844964 19561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cffe81cc5da4154b0c721b07d5d82d2" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38169 } }
I20260807 20:16:39.845060 19561 leader_election.cc:302] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cffe81cc5da4154b0c721b07d5d82d2; no voters:
I20260807 20:16:39.845269 19561 leader_election.cc:288] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:16:39.845311 19566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:39.845453 19566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [term 1 LEADER]: Becoming Leader. State: Replica: 1cffe81cc5da4154b0c721b07d5d82d2, State: Running, Role: LEADER
I20260807 20:16:39.845539 19561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:16:39.845583 19566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cffe81cc5da4154b0c721b07d5d82d2" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38169 } }
I20260807 20:16:39.846019 19570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cffe81cc5da4154b0c721b07d5d82d2. Latest consensus state: current_term: 1 leader_uuid: "1cffe81cc5da4154b0c721b07d5d82d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cffe81cc5da4154b0c721b07d5d82d2" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38169 } } }
I20260807 20:16:39.846139 19568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cffe81cc5da4154b0c721b07d5d82d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cffe81cc5da4154b0c721b07d5d82d2" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38169 } } }
I20260807 20:16:39.846169 19570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:39.846230 19568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:39.846836 19582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:16:39.846849 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 19446
I20260807 20:16:39.846956 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:16:39.847148 24355 external_mini_cluster.cc:1428] Setting key dfc4dcfaedf2a0cf4188b07bff68f56d
I20260807 20:16:39.847380 19582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:16:39.848963 19582 catalog_manager.cc:1347] Generated new cluster ID: 82c60a1a7526460da09bafa0e5b2f1c2
I20260807 20:16:39.849021 19582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:16:39 dist-test-slave-2j7r krb5kdc[19422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:16:39.861407 19582 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:16:39.861929 19582 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:16:39.866691 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:39.847621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54870 (local address 127.23.200.254:38169)
0807 20:16:39.852957 (+ 5336us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:39.852968 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:39.853010 (+ 42us) server_negotiation.cc:407] Connection header received
0807 20:16:39.853170 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:39.853177 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:39.853246 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:39.853403 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:39.854165 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:39.856057 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:39.857069 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:39.857713 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:39.857840 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:39.860470 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:39.860474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:39.860474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:39.860485 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:39.864933 (+ 4448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:39.865366 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:39.865370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:39.865370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:39.865498 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:39.865693 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:39.865694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:39.865694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:39.866012 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:39.866179 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:39.866262 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 20:16:39.866426 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5262,"thread_start_us":81,"threads_started":1}
I20260807 20:16:39.871743 19582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2: Generated new TSK 0
I20260807 20:16:39.871943 19582 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:16:39.912766 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:38169
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:39.997504 19606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:39.997709 19606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:39.997730 19606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:39.998862 19606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:39.998912 19606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:39.998937 19606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:16:40.000398 19606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:38169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:40.000620 19606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:40.000842 19606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:16:40.006139 19606 server_base.cc:1034] running on GCE node
W20260807 20:16:40.006119 19618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:40.006105 19616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:40.006115 19623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:40.006567 19606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:40.006825 19606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:40.007997 19606 hybrid_clock.cc:648] HybridClock initialized: now 1786133800007977 us; error 28 us; skew 500 ppm
Aug 07 20:16:40 dist-test-slave-2j7r krb5kdc[19422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133800, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:40.010831 19606 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:16:40.011277 19606 webserver.cc:460] Webserver started at http://127.23.200.193:46817/ using document root <none> and password file <none>
I20260807 20:16:40.011493 19606 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:40.011538 19606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:40.011642 19606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:40.012665 19606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "926e3bc454474d1e843a82ce037767c2"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "2d2023ccd79e719bbad240ca7c9f676b"
server_key_iv: "16411b282ce177cbb6b056110301e0d5"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.012991 19606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "926e3bc454474d1e843a82ce037767c2"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "2d2023ccd79e719bbad240ca7c9f676b"
server_key_iv: "16411b282ce177cbb6b056110301e0d5"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.014503 19606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:16:40.015266 19631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:40.015420 19606 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:16:40.015506 19606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "926e3bc454474d1e843a82ce037767c2"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "2d2023ccd79e719bbad240ca7c9f676b"
server_key_iv: "16411b282ce177cbb6b056110301e0d5"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.015574 19606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:40.025887 19606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:40.028273 19606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:40.028432 19606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:40.028657 19606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:40.028960 19606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:40.028995 19606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:40.029017 19606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:40.029040 19606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:40.034591 19606 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:44283
I20260807 20:16:40.034651 19819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:44283 every 8 connection(s)
I20260807 20:16:40.034982 19606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:16:40.038594 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 19606
I20260807 20:16:40.038695 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:16:40.038901 24355 external_mini_cluster.cc:1428] Setting key 070a09e6fdb45bb190f86ae056b54d41
Aug 07 20:16:40 dist-test-slave-2j7r krb5kdc[19422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133800, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:16:40.044478 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.035667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:40903 (local address 127.23.200.254:38169)
0807 20:16:40.035722 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:40.035724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:40.035928 (+ 204us) server_negotiation.cc:407] Connection header received
0807 20:16:40.036450 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:40.036451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:40.036463 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:40.036518 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:40.037554 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.038059 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.039093 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.039363 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.039435 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.041612 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:40.041613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:40.041614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:40.041622 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:40.042918 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:40.043391 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:40.043392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:40.043393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:40.043461 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:40.043771 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:40.043772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:40.043772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:40.043903 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:40.043975 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:40.044317 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 20:16:40.044410 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 20:16:40.044710 19824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.035633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:38169 (local address 127.23.200.193:40903)
0807 20:16:40.035811 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:40.035848 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:40.036345 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:40.036592 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:40.036597 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:40.037247 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:40.037458 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.037465 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.038144 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.038146 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.039012 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.039015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.039452 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.039453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.039602 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.040124 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:16:40.040133 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:40.041518 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:40.043004 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:40.043013 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:40.043015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:40.043298 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:40.043541 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:40.043542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:40.043542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:40.043703 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:40.043972 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:40.043980 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:40.044157 (+ 177us) client_negotiation.cc:769] Sending connection context
0807 20:16:40.044276 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 20:16:40.044467 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260807 20:16:40.045271 19820 heartbeater.cc:347] Connected to a master server at 127.23.200.254:38169
I20260807 20:16:40.045383 19820 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:40.045599 19820 heartbeater.cc:511] Master 127.23.200.254:38169 requested a full tablet report, sending...
I20260807 20:16:40.046214 19496 ts_manager.cc:194] Registered new tserver with Master: 926e3bc454474d1e843a82ce037767c2 (127.23.200.193:44283)
I20260807 20:16:40.046898 19496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:40903
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:16:40.068202 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:38169
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:40.150271 19831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:40.150477 19831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:40.150501 19831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:40.151608 19831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:40.151648 19831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:40.151675 19831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:16:40.153119 19831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:38169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:40.153357 19831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:40.153575 19831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:40.158368 19839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:40.158466 19838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:40.158370 19841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:40.158548 19831 server_base.cc:1034] running on GCE node
I20260807 20:16:40.158727 19831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:40.158938 19831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:40.160073 19831 hybrid_clock.cc:648] HybridClock initialized: now 1786133800160053 us; error 36 us; skew 500 ppm
Aug 07 20:16:40 dist-test-slave-2j7r krb5kdc[19422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133800, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:40.162473 19831 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:16:40.162911 19831 webserver.cc:460] Webserver started at http://127.23.200.194:34493/ using document root <none> and password file <none>
I20260807 20:16:40.163110 19831 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:40.163157 19831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:40.163280 19831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:40.164279 19831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "45a0d69871d448c4a725c4b8795bf679"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "fa4d1b632385ba8bf5c20f5aebed359e"
server_key_iv: "b42731c5a4872f12400303e5bc1f722c"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.164602 19831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "45a0d69871d448c4a725c4b8795bf679"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "fa4d1b632385ba8bf5c20f5aebed359e"
server_key_iv: "b42731c5a4872f12400303e5bc1f722c"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.165887 19831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:16:40.166601 19854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:40.166751 19831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:16:40.166839 19831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "45a0d69871d448c4a725c4b8795bf679"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "fa4d1b632385ba8bf5c20f5aebed359e"
server_key_iv: "b42731c5a4872f12400303e5bc1f722c"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.166896 19831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:40.182832 19831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:40.184684 19831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:40.184854 19831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:40.185065 19831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:40.185382 19831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:40.185415 19831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:40.185443 19831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:40.185460 19831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:40.191073 19831 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:36313
I20260807 20:16:40.191128 20071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:36313 every 8 connection(s)
I20260807 20:16:40.191411 19831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:16:40.193886 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 19831
I20260807 20:16:40.194000 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:16:40.194205 24355 external_mini_cluster.cc:1428] Setting key d067314909af90a1dfe82570c1c71fb4
Aug 07 20:16:40 dist-test-slave-2j7r krb5kdc[19422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133800, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:16:40.201541 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.192111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:60397 (local address 127.23.200.254:38169)
0807 20:16:40.192168 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:40.192170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:40.192373 (+ 203us) server_negotiation.cc:407] Connection header received
0807 20:16:40.192825 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:40.192826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:40.192836 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:40.192889 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:40.193888 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.194670 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.195654 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.195906 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.195977 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.198444 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:40.198446 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:40.198447 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:40.198454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:40.199928 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:40.200459 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:40.200460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:40.200460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:40.200520 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:40.200811 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:40.200811 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:40.200812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:40.200924 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:40.200982 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:40.201380 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 20:16:40.201458 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 20:16:40.201776 20076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.192085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:38169 (local address 127.23.200.194:60397)
0807 20:16:40.192248 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:40.192289 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:40.192753 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:40.192944 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:40.192947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:40.193551 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:40.193772 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.193777 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.194760 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.194761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.195560 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.195563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.195979 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.195979 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.196084 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.196671 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:16:40.196682 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:40.198331 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:40.200013 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:40.200023 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:40.200026 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:40.200337 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:40.200584 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:40.200585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:40.200586 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:40.200737 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:40.200987 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:40.200994 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:40.201204 (+ 210us) client_negotiation.cc:769] Sending connection context
0807 20:16:40.201333 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 20:16:40.201515 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260807 20:16:40.202448 20072 heartbeater.cc:347] Connected to a master server at 127.23.200.254:38169
I20260807 20:16:40.202558 20072 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:40.202762 20072 heartbeater.cc:511] Master 127.23.200.254:38169 requested a full tablet report, sending...
I20260807 20:16:40.203208 19496 ts_manager.cc:194] Registered new tserver with Master: 45a0d69871d448c4a725c4b8795bf679 (127.23.200.194:36313)
I20260807 20:16:40.203718 19496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:60397
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:16:40.224812 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:38169
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:39709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:40.307235 20083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:40.307447 20083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:40.307469 20083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:40.308532 20083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:40.308562 20083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:40.308586 20083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:16:40.309995 20083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:39709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:38169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:40.310189 20083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:40.310375 20083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:40.315342 20090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:40.315382 20094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:40.315557 20083 server_base.cc:1034] running on GCE node
W20260807 20:16:40.315382 20089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:40.315914 20083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:40.316155 20083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:40.317342 20083 hybrid_clock.cc:648] HybridClock initialized: now 1786133800317321 us; error 38 us; skew 500 ppm
Aug 07 20:16:40 dist-test-slave-2j7r krb5kdc[19422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133800, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:40.319988 20083 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:16:40.320497 20083 webserver.cc:460] Webserver started at http://127.23.200.195:42651/ using document root <none> and password file <none>
I20260807 20:16:40.320695 20083 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:40.320737 20083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:40.320871 20083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:40.322003 20083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "154c017d26fb4581b28ab320f421a71a"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "c3f253aafdbf9179ca6e7fa3bbf50502"
server_key_iv: "895c858eda4b152021175f9fd49342a7"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.322336 20083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "154c017d26fb4581b28ab320f421a71a"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "c3f253aafdbf9179ca6e7fa3bbf50502"
server_key_iv: "895c858eda4b152021175f9fd49342a7"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.323591 20083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:16:40.324281 20106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:40.324463 20083 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:16:40.324551 20083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "154c017d26fb4581b28ab320f421a71a"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "c3f253aafdbf9179ca6e7fa3bbf50502"
server_key_iv: "895c858eda4b152021175f9fd49342a7"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.324611 20083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:40.341043 20083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:40.343282 20083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:40.343418 20083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:40.343640 20083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:40.343938 20083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:40.343971 20083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:40.343993 20083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:40.344005 20083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:40.350175 20083 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:35959
I20260807 20:16:40.350242 20293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:35959 every 8 connection(s)
I20260807 20:16:40.350543 20083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
Aug 07 20:16:40 dist-test-slave-2j7r krb5kdc[19422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133800, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:16:40.359292 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.351209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:58145 (local address 127.23.200.254:38169)
0807 20:16:40.351291 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:40.351294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:40.351492 (+ 198us) server_negotiation.cc:407] Connection header received
0807 20:16:40.351995 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:40.351997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:40.352010 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:40.352064 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:40.353012 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.353631 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.354597 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.354804 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.354859 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.356923 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:40.356924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:40.356925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:40.356932 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:40.358117 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:40.358479 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:40.358480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:40.358480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:40.358528 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:40.358753 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:40.358755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:40.358755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:40.358878 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:40.358942 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:40.359145 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 20:16:40.359210 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 20:16:40.359421 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.351177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:38169 (local address 127.23.200.195:58145)
0807 20:16:40.351370 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:40.351408 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:40.351901 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:40.352134 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:40.352139 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:40.352720 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:40.352918 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.352923 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.353717 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.353718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.354514 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.354517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.354872 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.354873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.354991 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.355434 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:16:40.355443 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:40.356835 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:40.358184 (+ 1349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:40.358190 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:40.358192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:40.358419 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:40.358573 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:40.358574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:40.358574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:40.358663 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:40.358934 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:40.358936 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:40.359044 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 20:16:40.359120 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 20:16:40.359235 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":82,"threads_started":1}
I20260807 20:16:40.359848 20296 heartbeater.cc:347] Connected to a master server at 127.23.200.254:38169
I20260807 20:16:40.359943 20296 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:40.360114 20296 heartbeater.cc:511] Master 127.23.200.254:38169 requested a full tablet report, sending...
I20260807 20:16:40.360483 19496 ts_manager.cc:194] Registered new tserver with Master: 154c017d26fb4581b28ab320f421a71a (127.23.200.195:35959)
I20260807 20:16:40.360764 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 20083
I20260807 20:16:40.360863 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:16:40.360898 19496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:58145
I20260807 20:16:40.361124 24355 external_mini_cluster.cc:1428] Setting key e9d87980d795bb53e044558991df2f28
I20260807 20:16:40.362073 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:16:40.369617 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.363449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54886 (local address 127.23.200.254:38169)
0807 20:16:40.363517 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:40.363519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:40.363603 (+ 84us) server_negotiation.cc:407] Connection header received
0807 20:16:40.363642 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:40.363643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:40.363652 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:40.363696 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:40.364253 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.364717 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.365534 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.365723 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.365795 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.366987 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:40.366989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:40.366989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:40.366996 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:40.368378 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:40.368734 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:40.368735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:40.368735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:40.368796 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:40.368988 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:40.368988 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:40.368988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:40.369200 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:40.369282 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:40.369406 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 20:16:40.369534 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 20:16:40.371094 19496 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:16:40.372694 19496 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:16:40.379251 20317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.375467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48138 (local address 127.23.200.195:35959)
0807 20:16:40.375662 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:40.375666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:40.375675 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:16:40.375714 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:40.375716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:40.375732 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:40.375806 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:40.376012 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.377134 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.378378 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.378871 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.378936 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.378995 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:40.379128 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 20:16:40.379171 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":84,"threads_started":1}
I20260807 20:16:40.379269 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.375066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:36313 (local address 127.0.0.1:58968)
0807 20:16:40.375173 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:40.375214 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:40.375276 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:40.376034 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:40.376035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:40.376036 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:40.376065 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.376066 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.376871 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.376872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.378110 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.378113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.379030 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.379032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.379103 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.379110 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 20:16:40.379138 (+ 28us) client_negotiation.cc:240] Negotiation successful
0807 20:16:40.379187 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":9,"thread_start_us":52,"threads_started":1}
I20260807 20:16:40.379392 20314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.375072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:44283 (local address 127.0.0.1:39682)
0807 20:16:40.375478 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:40.375488 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:40.375550 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:40.375715 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:40.375719 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:40.375721 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:40.375794 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.375796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.377215 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.377216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.378160 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.378163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.379032 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.379032 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.379103 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.379109 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 20:16:40.379138 (+ 29us) client_negotiation.cc:240] Negotiation successful
0807 20:16:40.379187 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":58,"thread_start_us":82,"threads_started":1}
I20260807 20:16:40.379494 20315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.375271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39682 (local address 127.23.200.193:44283)
0807 20:16:40.375461 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:40.375467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:40.375555 (+ 88us) server_negotiation.cc:407] Connection header received
0807 20:16:40.375610 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:40.375613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:40.375630 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:40.375707 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:40.375924 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.377069 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.378258 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.378929 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.379248 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.379343 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:40.379377 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 20:16:40.379419 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":85,"threads_started":1}
I20260807 20:16:40.379652 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.375263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:35959 (local address 127.0.0.1:48138)
0807 20:16:40.375541 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:40.375552 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:40.375606 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:40.375890 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:40.375891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:40.375893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:40.375944 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.375946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.377238 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.377239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.378302 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.378304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.378944 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.378945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.379023 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.379038 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 20:16:40.379103 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 20:16:40.379160 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":69,"threads_started":1}
I20260807 20:16:40.379981 20316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.375367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58968 (local address 127.23.200.194:36313)
0807 20:16:40.375876 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:40.375880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:40.375890 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:16:40.375928 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:40.375930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:40.375944 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:40.376038 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:40.376113 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.376777 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.378196 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.378929 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.379744 (+ 815us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.379815 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:40.379841 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 20:16:40.379906 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":90,"threads_started":1}
I20260807 20:16:40.380164 19710 tablet_service.cc:1467] Processing CreateTablet for tablet d3c86ba76ceb4c0faa4d4cddfbe57d1c (DEFAULT_TABLE table=test-table [id=4b616d2e53ee43498223b78e439ce9fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:40.380494 19710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3c86ba76ceb4c0faa4d4cddfbe57d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:40.381750 19947 tablet_service.cc:1467] Processing CreateTablet for tablet d3c86ba76ceb4c0faa4d4cddfbe57d1c (DEFAULT_TABLE table=test-table [id=4b616d2e53ee43498223b78e439ce9fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:40.381829 20185 tablet_service.cc:1467] Processing CreateTablet for tablet d3c86ba76ceb4c0faa4d4cddfbe57d1c (DEFAULT_TABLE table=test-table [id=4b616d2e53ee43498223b78e439ce9fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:40.382105 19947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3c86ba76ceb4c0faa4d4cddfbe57d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:40.382141 20185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3c86ba76ceb4c0faa4d4cddfbe57d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:40.383876 20320 tablet_bootstrap.cc:492] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2: Bootstrap starting.
I20260807 20:16:40.384578 20320 tablet_bootstrap.cc:654] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:40.384974 20322 tablet_bootstrap.cc:492] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a: Bootstrap starting.
I20260807 20:16:40.385342 20320 log.cc:824] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:40.385551 20323 tablet_bootstrap.cc:492] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679: Bootstrap starting.
I20260807 20:16:40.385741 20322 tablet_bootstrap.cc:654] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:40.386093 20320 tablet_bootstrap.cc:492] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2: No bootstrap required, opened a new log
I20260807 20:16:40.386204 20320 ts_tablet_manager.cc:1397] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:16:40.386200 20323 tablet_bootstrap.cc:654] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:40.386255 20322 log.cc:824] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:40.386704 20323 log.cc:824] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:40.387434 20323 tablet_bootstrap.cc:492] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679: No bootstrap required, opened a new log
I20260807 20:16:40.387504 20323 ts_tablet_manager.cc:1397] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:16:40.387954 20320 raft_consensus.cc:348] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45a0d69871d448c4a725c4b8795bf679" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 36313 } } peers { permanent_uuid: "926e3bc454474d1e843a82ce037767c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44283 } } peers { permanent_uuid: "154c017d26fb4581b28ab320f421a71a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35959 } }
I20260807 20:16:40.388126 20320 raft_consensus.cc:374] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:40.388170 20320 raft_consensus.cc:729] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926e3bc454474d1e843a82ce037767c2, State: Initialized, Role: FOLLOWER
I20260807 20:16:40.388288 20320 consensus_queue.cc:260] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45a0d69871d448c4a725c4b8795bf679" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 36313 } } peers { permanent_uuid: "926e3bc454474d1e843a82ce037767c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44283 } } peers { permanent_uuid: "154c017d26fb4581b28ab320f421a71a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35959 } }
I20260807 20:16:40.388568 20320 ts_tablet_manager.cc:1428] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:16:40.388602 19820 heartbeater.cc:503] Master 127.23.200.254:38169 was elected leader, sending a full tablet report...
I20260807 20:16:40.389320 20323 raft_consensus.cc:348] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45a0d69871d448c4a725c4b8795bf679" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 36313 } } peers { permanent_uuid: "926e3bc454474d1e843a82ce037767c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44283 } } peers { permanent_uuid: "154c017d26fb4581b28ab320f421a71a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35959 } }
I20260807 20:16:40.389479 20322 tablet_bootstrap.cc:492] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a: No bootstrap required, opened a new log
I20260807 20:16:40.389480 20323 raft_consensus.cc:374] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:40.389515 20323 raft_consensus.cc:729] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45a0d69871d448c4a725c4b8795bf679, State: Initialized, Role: FOLLOWER
I20260807 20:16:40.389541 20322 ts_tablet_manager.cc:1397] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 20:16:40.389617 20323 consensus_queue.cc:260] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45a0d69871d448c4a725c4b8795bf679" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 36313 } } peers { permanent_uuid: "926e3bc454474d1e843a82ce037767c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44283 } } peers { permanent_uuid: "154c017d26fb4581b28ab320f421a71a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35959 } }
I20260807 20:16:40.389881 20323 ts_tablet_manager.cc:1428] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 20:16:40.389909 20072 heartbeater.cc:503] Master 127.23.200.254:38169 was elected leader, sending a full tablet report...
I20260807 20:16:40.391168 20322 raft_consensus.cc:348] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45a0d69871d448c4a725c4b8795bf679" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 36313 } } peers { permanent_uuid: "926e3bc454474d1e843a82ce037767c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44283 } } peers { permanent_uuid: "154c017d26fb4581b28ab320f421a71a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35959 } }
I20260807 20:16:40.391292 20322 raft_consensus.cc:374] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:40.391312 20322 raft_consensus.cc:729] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 154c017d26fb4581b28ab320f421a71a, State: Initialized, Role: FOLLOWER
I20260807 20:16:40.391397 20322 consensus_queue.cc:260] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45a0d69871d448c4a725c4b8795bf679" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 36313 } } peers { permanent_uuid: "926e3bc454474d1e843a82ce037767c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44283 } } peers { permanent_uuid: "154c017d26fb4581b28ab320f421a71a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35959 } }
I20260807 20:16:40.391628 20322 ts_tablet_manager.cc:1428] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:16:40.391666 20296 heartbeater.cc:503] Master 127.23.200.254:38169 was elected leader, sending a full tablet report...
W20260807 20:16:40.441952 20075 tablet.cc:2369] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:16:40.464314 20330 raft_consensus.cc:482] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:16:40.464455 20330 raft_consensus.cc:504] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45a0d69871d448c4a725c4b8795bf679" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 36313 } } peers { permanent_uuid: "926e3bc454474d1e843a82ce037767c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44283 } } peers { permanent_uuid: "154c017d26fb4581b28ab320f421a71a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35959 } }
I20260807 20:16:40.464851 20330 leader_election.cc:288] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45a0d69871d448c4a725c4b8795bf679 (127.23.200.194:36313), 154c017d26fb4581b28ab320f421a71a (127.23.200.195:35959)
I20260807 20:16:40.467793 20316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.465169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:55895 (local address 127.23.200.194:36313)
0807 20:16:40.465238 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:40.465240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:40.465252 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:16:40.465284 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:40.465284 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:40.465293 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:40.465413 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:40.465566 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.466048 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.467107 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.467485 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.467549 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.467558 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:40.467695 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 20:16:40.467731 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 20:16:40.467828 19824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.465047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:36313 (local address 127.23.200.193:55895)
0807 20:16:40.465143 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:40.465160 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:40.465229 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:40.465430 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:40.465431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:40.465433 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:40.465501 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.465503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.466126 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.466127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.467018 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.467021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.467558 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.467559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.467634 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.467637 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:16:40.467682 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 20:16:40.467723 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 20:16:40.467828 20317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.465364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:50075 (local address 127.23.200.195:35959)
0807 20:16:40.465431 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:40.465433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:40.465454 (+ 21us) server_negotiation.cc:407] Connection header received
0807 20:16:40.465480 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:40.465481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:40.465490 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:40.465532 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:40.465673 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.466201 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.467154 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.467524 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.467584 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.467593 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:40.467723 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 20:16:40.467763 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 20:16:40.467844 20334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.465179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:35959 (local address 127.23.200.193:50075)
0807 20:16:40.465347 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:40.465358 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:40.465413 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:40.465542 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:40.465543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:40.465544 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:40.465620 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.465622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.466272 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.466273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.467077 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.467079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.467595 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.467596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.467666 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.467671 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:16:40.467714 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 20:16:40.467752 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":121,"threads_started":1}
I20260807 20:16:40.468155 20222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3c86ba76ceb4c0faa4d4cddfbe57d1c" candidate_uuid: "926e3bc454474d1e843a82ce037767c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "154c017d26fb4581b28ab320f421a71a" is_pre_election: true
I20260807 20:16:40.468298 20222 raft_consensus.cc:2454] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 926e3bc454474d1e843a82ce037767c2 in term 0.
I20260807 20:16:40.468320 19986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3c86ba76ceb4c0faa4d4cddfbe57d1c" candidate_uuid: "926e3bc454474d1e843a82ce037767c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45a0d69871d448c4a725c4b8795bf679" is_pre_election: true
I20260807 20:16:40.468479 19986 raft_consensus.cc:2454] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 926e3bc454474d1e843a82ce037767c2 in term 0.
I20260807 20:16:40.468554 19633 leader_election.cc:302] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 154c017d26fb4581b28ab320f421a71a, 926e3bc454474d1e843a82ce037767c2; no voters:
I20260807 20:16:40.468691 20330 raft_consensus.cc:2781] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:16:40.468765 20330 raft_consensus.cc:482] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:16:40.468782 20330 raft_consensus.cc:3037] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:40.469581 20330 raft_consensus.cc:504] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45a0d69871d448c4a725c4b8795bf679" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 36313 } } peers { permanent_uuid: "926e3bc454474d1e843a82ce037767c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44283 } } peers { permanent_uuid: "154c017d26fb4581b28ab320f421a71a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35959 } }
I20260807 20:16:40.469698 20330 leader_election.cc:288] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [CANDIDATE]: Term 1 election: Requested vote from peers 45a0d69871d448c4a725c4b8795bf679 (127.23.200.194:36313), 154c017d26fb4581b28ab320f421a71a (127.23.200.195:35959)
I20260807 20:16:40.469910 20222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3c86ba76ceb4c0faa4d4cddfbe57d1c" candidate_uuid: "926e3bc454474d1e843a82ce037767c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "154c017d26fb4581b28ab320f421a71a"
I20260807 20:16:40.469925 19986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3c86ba76ceb4c0faa4d4cddfbe57d1c" candidate_uuid: "926e3bc454474d1e843a82ce037767c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45a0d69871d448c4a725c4b8795bf679"
I20260807 20:16:40.469990 19986 raft_consensus.cc:3037] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:40.469983 20222 raft_consensus.cc:3037] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:40.470700 20222 raft_consensus.cc:2454] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 926e3bc454474d1e843a82ce037767c2 in term 1.
I20260807 20:16:40.470718 19986 raft_consensus.cc:2454] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 926e3bc454474d1e843a82ce037767c2 in term 1.
I20260807 20:16:40.470889 19633 leader_election.cc:302] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45a0d69871d448c4a725c4b8795bf679, 926e3bc454474d1e843a82ce037767c2; no voters:
I20260807 20:16:40.470986 20330 raft_consensus.cc:2781] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:40.471148 20330 raft_consensus.cc:686] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [term 1 LEADER]: Becoming Leader. State: Replica: 926e3bc454474d1e843a82ce037767c2, State: Running, Role: LEADER
I20260807 20:16:40.471256 20330 consensus_queue.cc:237] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45a0d69871d448c4a725c4b8795bf679" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 36313 } } peers { permanent_uuid: "926e3bc454474d1e843a82ce037767c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44283 } } peers { permanent_uuid: "154c017d26fb4581b28ab320f421a71a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35959 } }
I20260807 20:16:40.472129 19496 catalog_manager.cc:5495] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 926e3bc454474d1e843a82ce037767c2 (127.23.200.193). New cstate: current_term: 1 leader_uuid: "926e3bc454474d1e843a82ce037767c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45a0d69871d448c4a725c4b8795bf679" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 36313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "926e3bc454474d1e843a82ce037767c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 44283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "154c017d26fb4581b28ab320f421a71a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35959 } health_report { overall_health: UNKNOWN } } }
I20260807 20:16:40.494005 20315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.491669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39686 (local address 127.23.200.193:44283)
0807 20:16:40.491721 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:40.491723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:40.491732 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:16:40.491780 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:40.491781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:40.491803 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:40.491863 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:16:40.491999 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.492509 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.493294 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.493484 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.493539 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.493695 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:16:40.493803 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:16:40.493873 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:40.493908 (+ 35us) server_negotiation.cc:299] Negotiation successful
0807 20:16:40.493928 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 20:16:40.495782 19986 raft_consensus.cc:1261] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679 [term 1 FOLLOWER]: Refusing update from remote peer 926e3bc454474d1e843a82ce037767c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:16:40.495803 20222 raft_consensus.cc:1261] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a [term 1 FOLLOWER]: Refusing update from remote peer 926e3bc454474d1e843a82ce037767c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:16:40.496131 20335 consensus_queue.cc:1035] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45a0d69871d448c4a725c4b8795bf679" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 36313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:16:40.496260 20330 consensus_queue.cc:1035] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 926e3bc454474d1e843a82ce037767c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "154c017d26fb4581b28ab320f421a71a" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 35959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:16:40.498994 20341 mvcc.cc:205] Tried to move back new op lower bound from 7316004046828019712 to 7316004046730838016. Current Snapshot: MvccSnapshot[applied={T|T < 7316004046828019712}]
I20260807 20:16:40.501123 19495 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:16:40.501226 19495 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:16:40.501935 19495 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1cffe81cc5da4154b0c721b07d5d82d2: Sending DeleteTablet for 3 replicas of tablet d3c86ba76ceb4c0faa4d4cddfbe57d1c
I20260807 20:16:40.502228 19947 tablet_service.cc:1514] Processing DeleteTablet for tablet d3c86ba76ceb4c0faa4d4cddfbe57d1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:40 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:58968
I20260807 20:16:40.502265 19710 tablet_service.cc:1514] Processing DeleteTablet for tablet d3c86ba76ceb4c0faa4d4cddfbe57d1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:40 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:39682
I20260807 20:16:40.502290 20185 tablet_service.cc:1514] Processing DeleteTablet for tablet d3c86ba76ceb4c0faa4d4cddfbe57d1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:40 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:48138
I20260807 20:16:40.502621 20360 tablet_replica.cc:330] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a: stopping tablet replica
I20260807 20:16:40.502650 20357 tablet_replica.cc:330] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679: stopping tablet replica
I20260807 20:16:40.502717 20360 raft_consensus.cc:2229] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:16:40.502745 20357 raft_consensus.cc:2229] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:16:40.502812 20360 raft_consensus.cc:2258] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:40.502846 20357 raft_consensus.cc:2258] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:40.502885 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 19606
I20260807 20:16:40.506049 20357 ts_tablet_manager.cc:1905] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:16:40.507972 20357 ts_tablet_manager.cc:1918] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:16:40.508049 20357 log.cc:1196] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/d3c86ba76ceb4c0faa4d4cddfbe57d1c
I20260807 20:16:40.508360 20357 ts_tablet_manager.cc:1939] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 45a0d69871d448c4a725c4b8795bf679: Deleting consensus metadata
I20260807 20:16:40.508817 19470 catalog_manager.cc:4841] TS 45a0d69871d448c4a725c4b8795bf679 (127.23.200.194:36313): tablet d3c86ba76ceb4c0faa4d4cddfbe57d1c (table test-table [id=4b616d2e53ee43498223b78e439ce9fd]) successfully deleted
I20260807 20:16:40.511358 20360 ts_tablet_manager.cc:1905] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:16:40.512781 20360 ts_tablet_manager.cc:1918] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:16:40.512840 20360 log.cc:1196] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/d3c86ba76ceb4c0faa4d4cddfbe57d1c
I20260807 20:16:40.513049 20360 ts_tablet_manager.cc:1939] T d3c86ba76ceb4c0faa4d4cddfbe57d1c P 154c017d26fb4581b28ab320f421a71a: Deleting consensus metadata
I20260807 20:16:40.513456 19470 catalog_manager.cc:4841] TS 154c017d26fb4581b28ab320f421a71a (127.23.200.195:35959): tablet d3c86ba76ceb4c0faa4d4cddfbe57d1c (table test-table [id=4b616d2e53ee43498223b78e439ce9fd]) successfully deleted
W20260807 20:16:40.519356 19476 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:44283 (error 108)
I20260807 20:16:40.519534 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 19831
I20260807 20:16:40.526961 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.526797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:44283 (local address 127.0.0.1:39688)
0807 20:16:40.526856 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:40.526916 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:44283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 20:16:40.527112 19476 catalog_manager.cc:4584] TS 926e3bc454474d1e843a82ce037767c2 (127.23.200.193:44283): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d3c86ba76ceb4c0faa4d4cddfbe57d1c: Network error: Client connection negotiation failed: client connection to 127.23.200.193:44283: connect: Connection refused (error 111)
I20260807 20:16:40.530591 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 20083
I20260807 20:16:40.539222 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 19446
2026-08-07T20:16:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2134 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T20:16:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:16:40Z Disabled control of system clock
I20260807 20:16:40.574745 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:40127
--webserver_interface=127.23.200.254
--webserver_port=0
--builtin_ntp_servers=127.23.200.212:35811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:40127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:16:40.661631 20367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:40.661849 20367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:40.661870 20367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:40.663021 20367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:16:40.663058 20367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:40.663070 20367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:40.663080 20367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:16:40.663091 20367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:16:40.664420 20367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:35811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:40127
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:40127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:40.664620 20367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:40.664875 20367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:40.669726 20383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:40.669802 20377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:40.669835 20375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:40.669865 20367 server_base.cc:1034] running on GCE node
I20260807 20:16:40.670215 20367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:40.670499 20367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:40.671640 20367 hybrid_clock.cc:648] HybridClock initialized: now 1786133800671625 us; error 33 us; skew 500 ppm
I20260807 20:16:40.672813 20367 webserver.cc:460] Webserver started at http://127.23.200.254:46713/ using document root <none> and password file <none>
I20260807 20:16:40.673050 20367 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:40.673126 20367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:40.673254 20367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:40.674328 20367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "b5311c8367f14040be44bd21bc2bcfe7"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "d010b7d9bbc542aeebdd1c939d392e8a"
server_key_iv: "2e7fdc293db63a02c48bc85676506fab"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.674669 20367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "b5311c8367f14040be44bd21bc2bcfe7"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "d010b7d9bbc542aeebdd1c939d392e8a"
server_key_iv: "2e7fdc293db63a02c48bc85676506fab"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.675894 20367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:16:40.676555 20404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:40.676716 20367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:16:40.676801 20367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "b5311c8367f14040be44bd21bc2bcfe7"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "d010b7d9bbc542aeebdd1c939d392e8a"
server_key_iv: "2e7fdc293db63a02c48bc85676506fab"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.676862 20367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:40.690344 20367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:40.690656 20367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:40.690785 20367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:40.694630 20367 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:40127
I20260807 20:16:40.694666 20493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:40127 every 8 connection(s)
I20260807 20:16:40.694990 20367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:16:40.695998 20495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:40.698472 20495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7: Bootstrap starting.
I20260807 20:16:40.699054 20495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:40.699330 20495 log.cc:824] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:40.699792 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 20367
I20260807 20:16:40.699895 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:16:40.700058 20495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7: No bootstrap required, opened a new log
I20260807 20:16:40.700073 24355 external_mini_cluster.cc:1428] Setting key fa3a9df391ef6884c1f736b9b71304a0
I20260807 20:16:40.702617 20495 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5311c8367f14040be44bd21bc2bcfe7" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 40127 } }
I20260807 20:16:40.702796 20495 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:40.702840 20495 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5311c8367f14040be44bd21bc2bcfe7, State: Initialized, Role: FOLLOWER
I20260807 20:16:40.702968 20495 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5311c8367f14040be44bd21bc2bcfe7" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 40127 } }
I20260807 20:16:40.703044 20495 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:16:40.703083 20495 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:16:40.703126 20495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:40.703789 20495 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5311c8367f14040be44bd21bc2bcfe7" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 40127 } }
I20260807 20:16:40.703890 20495 leader_election.cc:302] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5311c8367f14040be44bd21bc2bcfe7; no voters:
I20260807 20:16:40.704051 20495 leader_election.cc:288] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:16:40.704094 20504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:40.704253 20504 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [term 1 LEADER]: Becoming Leader. State: Replica: b5311c8367f14040be44bd21bc2bcfe7, State: Running, Role: LEADER
I20260807 20:16:40.704316 20495 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:16:40.704339 20504 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5311c8367f14040be44bd21bc2bcfe7" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 40127 } }
I20260807 20:16:40.705054 20506 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5311c8367f14040be44bd21bc2bcfe7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5311c8367f14040be44bd21bc2bcfe7" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 40127 } } }
I20260807 20:16:40.705310 20506 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:40.705241 20509 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5311c8367f14040be44bd21bc2bcfe7. Latest consensus state: current_term: 1 leader_uuid: "b5311c8367f14040be44bd21bc2bcfe7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5311c8367f14040be44bd21bc2bcfe7" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 40127 } } }
I20260807 20:16:40.705556 20509 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:40.706143 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.700616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56474 (local address 127.23.200.254:40127)
0807 20:16:40.700803 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:40.700810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:40.700836 (+ 26us) server_negotiation.cc:407] Connection header received
0807 20:16:40.701323 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:40.701330 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:40.701393 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:40.701504 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:40.701742 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.702592 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.703744 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.704337 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.704484 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.705229 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:40.705234 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:40.705234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:40.705458 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:40.705505 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:40.705522 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:40.705587 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:40.705850 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 20:16:40.705939 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
W20260807 20:16:40.706311 20527 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:16:40.706395 20527 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:16:40.706475 20520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:16:40.706662 20520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:16:40.708114 20520 catalog_manager.cc:1347] Generated new cluster ID: 6e0704daf31b46cb9ddf63c649049568
I20260807 20:16:40.708170 20520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:16:40.726461 20520 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:16:40.727192 20520 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:16:40.732949 20520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7: Generated new TSK 0
I20260807 20:16:40.733151 20520 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:16:40.734776 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:40127
--builtin_ntp_servers=127.23.200.212:35811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:16:40.835594 20532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:40.835807 20532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:40.835839 20532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:40.836968 20532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:40.837006 20532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:40.837039 20532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:16:40.838416 20532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:35811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:40127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:40.838662 20532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:40.838989 20532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:40.844025 20546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:40.844038 20543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:40.844035 20542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:40.844610 20532 server_base.cc:1034] running on GCE node
I20260807 20:16:40.844831 20532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:40.845058 20532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:40.846285 20532 hybrid_clock.cc:648] HybridClock initialized: now 1786133800846268 us; error 32 us; skew 500 ppm
I20260807 20:16:40.847582 20532 webserver.cc:460] Webserver started at http://127.23.200.193:44131/ using document root <none> and password file <none>
I20260807 20:16:40.847844 20532 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:40.847903 20532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:40.848030 20532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:40.849071 20532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "e49e56ecdb3840db8078b818692ee573"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "289daa541e2d76813135920194c6a210"
server_key_iv: "1975f41dc6f23087e94eca82b0e45482"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.849454 20532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "e49e56ecdb3840db8078b818692ee573"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "289daa541e2d76813135920194c6a210"
server_key_iv: "1975f41dc6f23087e94eca82b0e45482"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.850656 20532 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:16:40.851377 20560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:40.851539 20532 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:16:40.851626 20532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "e49e56ecdb3840db8078b818692ee573"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "289daa541e2d76813135920194c6a210"
server_key_iv: "1975f41dc6f23087e94eca82b0e45482"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.851689 20532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:40.864005 20532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:40.864291 20532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:40.864437 20532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:40.864672 20532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:40.865077 20532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:40.865146 20532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:40.865176 20532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:40.865236 20532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:40.871667 20532 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:45521
I20260807 20:16:40.871718 20740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:45521 every 8 connection(s)
I20260807 20:16:40.872061 20532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:16:40.877455 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.872762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:59283 (local address 127.23.200.254:40127)
0807 20:16:40.872818 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:40.872820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:40.873054 (+ 234us) server_negotiation.cc:407] Connection header received
0807 20:16:40.873626 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:40.873627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:40.873645 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:40.873710 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:40.874400 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.874912 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.875994 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.876196 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.876312 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.877206 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:40.877207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:40.877208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:40.877212 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:40.877226 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:40.877230 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:40.877269 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:40.877330 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 20:16:40.877396 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:16:40.877635 20746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.872740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:40127 (local address 127.23.200.193:59283)
0807 20:16:40.872929 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:40.872971 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:40.873509 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:40.873776 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:40.873779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:40.873788 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:40.874303 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.874307 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.875075 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.875076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.875900 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:40.875906 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:40.876329 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:40.876330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:40.876461 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:40.877016 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:16:40.877016 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:40.877056 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:16:40.877108 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:40.877275 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:40.877278 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:40.877281 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:16:40.877329 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 20:16:40.877436 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260807 20:16:40.878194 20741 heartbeater.cc:347] Connected to a master server at 127.23.200.254:40127
I20260807 20:16:40.878285 20741 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:40.878472 20741 heartbeater.cc:511] Master 127.23.200.254:40127 requested a full tablet report, sending...
I20260807 20:16:40.879194 20434 ts_manager.cc:194] Registered new tserver with Master: e49e56ecdb3840db8078b818692ee573 (127.23.200.193:45521)
I20260807 20:16:40.879911 20434 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.200.193:59283
I20260807 20:16:40.881407 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 20532
I20260807 20:16:40.881488 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:16:40.881645 24355 external_mini_cluster.cc:1428] Setting key 02b7807e34075cab1b1fb82bbeec883a
I20260807 20:16:40.882483 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:40127
--builtin_ntp_servers=127.23.200.212:35811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:16:40.965260 20748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:40.965477 20748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:40.965507 20748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:40.966687 20748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:40.966724 20748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:40.966758 20748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:16:40.968102 20748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:35811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:40127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:40.968339 20748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:40.968618 20748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:40.973568 20758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:40.973614 20755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:40.973619 20748 server_base.cc:1034] running on GCE node
W20260807 20:16:40.973624 20754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:40.973979 20748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:40.974217 20748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:40.975356 20748 hybrid_clock.cc:648] HybridClock initialized: now 1786133800975346 us; error 31 us; skew 500 ppm
I20260807 20:16:40.976550 20748 webserver.cc:460] Webserver started at http://127.23.200.194:36777/ using document root <none> and password file <none>
I20260807 20:16:40.976794 20748 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:40.976841 20748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:40.977005 20748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:40.978071 20748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "835c484325134201ad6ecad0451f5d1d"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "967272a2bd2fa50f0e866cc7fc75cc42"
server_key_iv: "48c4e439871e830c6442ef1e74aef759"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.978410 20748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "835c484325134201ad6ecad0451f5d1d"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "967272a2bd2fa50f0e866cc7fc75cc42"
server_key_iv: "48c4e439871e830c6442ef1e74aef759"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.979640 20748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:16:40.980398 20767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:40.980587 20748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:16:40.980677 20748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "835c484325134201ad6ecad0451f5d1d"
format_stamp: "Formatted at 2026-08-07 20:16:40 on dist-test-slave-2j7r"
server_key: "967272a2bd2fa50f0e866cc7fc75cc42"
server_key_iv: "48c4e439871e830c6442ef1e74aef759"
server_key_version: "encryptionkey@0"
I20260807 20:16:40.980786 20748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:40.992165 20748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:40.992452 20748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:40.992580 20748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:40.992781 20748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:40.993144 20748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:40.993181 20748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:40.993211 20748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:40.993234 20748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:40.998644 20748 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:43253
I20260807 20:16:40.998697 20949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:43253 every 8 connection(s)
I20260807 20:16:40.999059 20748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:16:41.003981 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.999730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:55439 (local address 127.23.200.254:40127)
0807 20:16:40.999813 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:40.999816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:41.000023 (+ 207us) server_negotiation.cc:407] Connection header received
0807 20:16:41.000512 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:41.000513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:41.000524 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:41.000577 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:41.001289 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.001770 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.002729 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.002936 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.003007 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:41.003748 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:41.003750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:41.003750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:41.003756 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:41.003772 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:41.003776 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:41.003828 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:41.003892 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 20:16:41.003916 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 20:16:41.004153 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:40.999723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:40127 (local address 127.23.200.194:55439)
0807 20:16:40.999908 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:40.999942 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:41.000427 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:41.000637 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:41.000640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:41.000649 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:41.001204 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:41.001210 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.001839 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.001839 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:41.002616 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:41.002618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.003008 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.003008 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:41.003108 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:41.003596 (+ 488us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:16:41.003597 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:41.003625 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:16:41.003648 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:41.003832 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:41.003835 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:41.003838 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:16:41.003887 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 20:16:41.003965 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260807 20:16:41.004685 20951 heartbeater.cc:347] Connected to a master server at 127.23.200.254:40127
I20260807 20:16:41.004801 20951 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:41.004997 20951 heartbeater.cc:511] Master 127.23.200.254:40127 requested a full tablet report, sending...
I20260807 20:16:41.005451 20434 ts_manager.cc:194] Registered new tserver with Master: 835c484325134201ad6ecad0451f5d1d (127.23.200.194:43253)
I20260807 20:16:41.005846 20434 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.200.194:55439
I20260807 20:16:41.008014 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 20748
I20260807 20:16:41.008103 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:16:41.008256 24355 external_mini_cluster.cc:1428] Setting key bc58588897058f2524ac46edd65fe668
I20260807 20:16:41.009068 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:40127
--builtin_ntp_servers=127.23.200.212:35811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:16:41.091703 20956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:41.091903 20956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:41.091924 20956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:41.092974 20956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:41.093004 20956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:41.093026 20956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:16:41.094368 20956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:35811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:40127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:41.094564 20956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:41.094816 20956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:41.099777 20963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:41.099777 20966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:41.099804 20962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:41.099830 20956 server_base.cc:1034] running on GCE node
I20260807 20:16:41.100185 20956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:41.100415 20956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:41.101578 20956 hybrid_clock.cc:648] HybridClock initialized: now 1786133801101565 us; error 32 us; skew 500 ppm
I20260807 20:16:41.102717 20956 webserver.cc:460] Webserver started at http://127.23.200.195:46037/ using document root <none> and password file <none>
I20260807 20:16:41.102972 20956 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:41.103019 20956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:41.103129 20956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:41.104159 20956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "41dd4182b050455e919a2207ba8ee33d"
format_stamp: "Formatted at 2026-08-07 20:16:41 on dist-test-slave-2j7r"
server_key: "c08f8482f562ca48c2532edb46f8a5d8"
server_key_iv: "536da681d7677769759d130c325280a4"
server_key_version: "encryptionkey@0"
I20260807 20:16:41.104516 20956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "41dd4182b050455e919a2207ba8ee33d"
format_stamp: "Formatted at 2026-08-07 20:16:41 on dist-test-slave-2j7r"
server_key: "c08f8482f562ca48c2532edb46f8a5d8"
server_key_iv: "536da681d7677769759d130c325280a4"
server_key_version: "encryptionkey@0"
I20260807 20:16:41.105867 20956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:16:41.106681 20974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:41.106827 20956 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:41.106916 20956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "41dd4182b050455e919a2207ba8ee33d"
format_stamp: "Formatted at 2026-08-07 20:16:41 on dist-test-slave-2j7r"
server_key: "c08f8482f562ca48c2532edb46f8a5d8"
server_key_iv: "536da681d7677769759d130c325280a4"
server_key_version: "encryptionkey@0"
I20260807 20:16:41.106989 20956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:41.126241 20956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:41.126555 20956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:41.126684 20956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:41.126904 20956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:41.127230 20956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:41.127264 20956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:41.127292 20956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:41.127310 20956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:41.132802 20956 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:34477
I20260807 20:16:41.132913 21160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:34477 every 8 connection(s)
I20260807 20:16:41.133217 20956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:16:41.134667 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 20956
I20260807 20:16:41.134766 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:16:41.134950 24355 external_mini_cluster.cc:1428] Setting key eaa5aea8df48e062e87904f16cd28ff2
I20260807 20:16:41.138517 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:41.133939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:34143 (local address 127.23.200.254:40127)
0807 20:16:41.133999 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:41.134001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:41.134210 (+ 209us) server_negotiation.cc:407] Connection header received
0807 20:16:41.134721 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:41.134723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:41.134736 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:41.134822 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:41.135716 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.136248 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.137240 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.137431 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.137487 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:41.138268 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:41.138269 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:41.138270 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:41.138275 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:41.138290 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:41.138294 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:41.138335 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:41.138410 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 20:16:41.138437 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:16:41.138717 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:41.133911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:40127 (local address 127.23.200.195:34143)
0807 20:16:41.134090 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:41.134123 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:41.134616 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:41.134923 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:41.134933 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:41.134942 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:41.135629 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:41.135635 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.136338 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.136339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:41.137162 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:41.137165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.137502 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.137504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:41.137640 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:41.138132 (+ 492us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:16:41.138133 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:41.138162 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:16:41.138186 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:41.138342 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:41.138347 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:41.138351 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:16:41.138414 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 20:16:41.138517 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260807 20:16:41.139117 21161 heartbeater.cc:347] Connected to a master server at 127.23.200.254:40127
I20260807 20:16:41.139209 21161 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:41.139390 21161 heartbeater.cc:511] Master 127.23.200.254:40127 requested a full tablet report, sending...
I20260807 20:16:41.139787 20434 ts_manager.cc:194] Registered new tserver with Master: 41dd4182b050455e919a2207ba8ee33d (127.23.200.195:34477)
I20260807 20:16:41.140147 20434 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.200.195:34143
I20260807 20:16:41.146663 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:16:41.150660 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:41.147953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56488 (local address 127.23.200.254:40127)
0807 20:16:41.148009 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:41.148011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:41.148097 (+ 86us) server_negotiation.cc:407] Connection header received
0807 20:16:41.148146 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:41.148147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:41.148161 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:41.148196 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:41.148401 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.148879 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.149683 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.149878 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.149929 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:41.150479 (+ 550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:41.150480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:41.150481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:41.150486 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:41.150500 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:41.150504 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:41.150540 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:41.150579 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 20:16:41.150599 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 20:16:41.152155 20434 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56488:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:16:41.153749 20434 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:16:41.160259 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:41.156042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:34477 (local address 127.0.0.1:39426)
0807 20:16:41.156383 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:41.156427 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:41.156499 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:41.156803 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:41.156807 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:41.156809 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:41.156875 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:41.156878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.157870 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.157871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:41.159028 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:41.159031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.159965 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.159966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:41.160041 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:41.160058 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 20:16:41.160119 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 20:16:41.160181 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":111,"threads_started":1}
I20260807 20:16:41.160279 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:41.156340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39426 (local address 127.23.200.195:34477)
0807 20:16:41.156521 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:41.156526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:41.156535 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:16:41.156571 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:41.156574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:41.156655 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:41.156727 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:41.156951 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.157795 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.159116 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.159871 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.159953 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:41.160042 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:41.160145 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 20:16:41.160189 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":78,"threads_started":1}
I20260807 20:16:41.160418 21174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:41.156047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:45521 (local address 127.0.0.1:58518)
0807 20:16:41.156597 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:41.156764 (+ 167us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:41.156846 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:41.157137 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:41.157138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:41.157140 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:41.157176 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:41.157177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.158422 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.158423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:41.159410 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:41.159412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.160212 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.160214 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:41.160271 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:41.160275 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:16:41.160321 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 20:16:41.160357 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":22,"thread_start_us":105,"threads_started":1}
I20260807 20:16:41.160696 21179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:41.156135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58518 (local address 127.23.200.193:45521)
0807 20:16:41.156322 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:41.156327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:41.156819 (+ 492us) server_negotiation.cc:407] Connection header received
0807 20:16:41.156952 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:41.156955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:41.156970 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:41.157023 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:41.157254 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.158327 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.159484 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.160015 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.160084 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:41.160179 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:41.160561 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 20:16:41.160608 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":74,"threads_started":1}
I20260807 20:16:41.161034 21178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:41.156042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:43253 (local address 127.0.0.1:53084)
0807 20:16:41.156428 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:41.156439 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:41.156502 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:41.156925 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:41.156926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:41.156929 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:41.156975 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:41.156977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.158390 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.158391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:41.159507 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:41.159510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.160787 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.160788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:41.160853 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:41.160858 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:16:41.160921 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 20:16:41.160962 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":2,"thread_start_us":96,"threads_started":1}
I20260807 20:16:41.161131 21049 tablet_service.cc:1467] Processing CreateTablet for tablet 8557adadfdb14c06920433ab8c4c0ca5 (DEFAULT_TABLE table=test-table [id=2ce90c0e4c35478da5fd3d2c18036a04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:41.161369 21181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:41.156388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53084 (local address 127.23.200.194:43253)
0807 20:16:41.156593 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:41.156598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:41.156610 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:16:41.156745 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:41.156748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:41.156766 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:41.156846 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:41.157058 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.158183 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.159619 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.160591 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.160667 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:41.160748 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:41.161240 (+ 492us) server_negotiation.cc:299] Negotiation successful
0807 20:16:41.161286 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":89,"threads_started":1}
I20260807 20:16:41.161345 20638 tablet_service.cc:1467] Processing CreateTablet for tablet 8557adadfdb14c06920433ab8c4c0ca5 (DEFAULT_TABLE table=test-table [id=2ce90c0e4c35478da5fd3d2c18036a04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:41.161550 21049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8557adadfdb14c06920433ab8c4c0ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:41.161707 20638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8557adadfdb14c06920433ab8c4c0ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:41.164183 21182 tablet_bootstrap.cc:492] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d: Bootstrap starting.
I20260807 20:16:41.164212 21184 tablet_bootstrap.cc:492] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573: Bootstrap starting.
I20260807 20:16:41.164914 21182 tablet_bootstrap.cc:654] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:41.164813 20849 tablet_service.cc:1467] Processing CreateTablet for tablet 8557adadfdb14c06920433ab8c4c0ca5 (DEFAULT_TABLE table=test-table [id=2ce90c0e4c35478da5fd3d2c18036a04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:41.165171 20849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8557adadfdb14c06920433ab8c4c0ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:41.165215 21182 log.cc:824] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:41.165627 21184 tablet_bootstrap.cc:654] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:41.165894 21182 tablet_bootstrap.cc:492] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d: No bootstrap required, opened a new log
I20260807 20:16:41.165963 21182 ts_tablet_manager.cc:1397] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:16:41.165992 21184 log.cc:824] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:41.166828 21184 tablet_bootstrap.cc:492] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573: No bootstrap required, opened a new log
I20260807 20:16:41.166909 21184 ts_tablet_manager.cc:1397] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 20:16:41.168098 21182 raft_consensus.cc:348] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41dd4182b050455e919a2207ba8ee33d" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34477 } } peers { permanent_uuid: "e49e56ecdb3840db8078b818692ee573" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45521 } } peers { permanent_uuid: "835c484325134201ad6ecad0451f5d1d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 43253 } }
I20260807 20:16:41.168231 21182 raft_consensus.cc:374] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:41.168262 21182 raft_consensus.cc:729] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41dd4182b050455e919a2207ba8ee33d, State: Initialized, Role: FOLLOWER
I20260807 20:16:41.168398 21182 consensus_queue.cc:260] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41dd4182b050455e919a2207ba8ee33d" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34477 } } peers { permanent_uuid: "e49e56ecdb3840db8078b818692ee573" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45521 } } peers { permanent_uuid: "835c484325134201ad6ecad0451f5d1d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 43253 } }
I20260807 20:16:41.168660 21182 ts_tablet_manager.cc:1428] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:16:41.168689 21187 tablet_bootstrap.cc:492] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d: Bootstrap starting.
I20260807 20:16:41.168802 21161 heartbeater.cc:503] Master 127.23.200.254:40127 was elected leader, sending a full tablet report...
I20260807 20:16:41.169402 21187 tablet_bootstrap.cc:654] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:41.169410 21184 raft_consensus.cc:348] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41dd4182b050455e919a2207ba8ee33d" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34477 } } peers { permanent_uuid: "e49e56ecdb3840db8078b818692ee573" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45521 } } peers { permanent_uuid: "835c484325134201ad6ecad0451f5d1d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 43253 } }
I20260807 20:16:41.169524 21184 raft_consensus.cc:374] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:41.169549 21184 raft_consensus.cc:729] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e49e56ecdb3840db8078b818692ee573, State: Initialized, Role: FOLLOWER
I20260807 20:16:41.169709 21187 log.cc:824] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:41.169674 21184 consensus_queue.cc:260] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41dd4182b050455e919a2207ba8ee33d" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34477 } } peers { permanent_uuid: "e49e56ecdb3840db8078b818692ee573" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45521 } } peers { permanent_uuid: "835c484325134201ad6ecad0451f5d1d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 43253 } }
I20260807 20:16:41.169943 21184 ts_tablet_manager.cc:1428] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:16:41.170300 20741 heartbeater.cc:503] Master 127.23.200.254:40127 was elected leader, sending a full tablet report...
I20260807 20:16:41.170368 21187 tablet_bootstrap.cc:492] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d: No bootstrap required, opened a new log
I20260807 20:16:41.170444 21187 ts_tablet_manager.cc:1397] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:16:41.172247 21187 raft_consensus.cc:348] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41dd4182b050455e919a2207ba8ee33d" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34477 } } peers { permanent_uuid: "e49e56ecdb3840db8078b818692ee573" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45521 } } peers { permanent_uuid: "835c484325134201ad6ecad0451f5d1d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 43253 } }
I20260807 20:16:41.172365 21187 raft_consensus.cc:374] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:41.172408 21187 raft_consensus.cc:729] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 835c484325134201ad6ecad0451f5d1d, State: Initialized, Role: FOLLOWER
I20260807 20:16:41.172514 21187 consensus_queue.cc:260] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41dd4182b050455e919a2207ba8ee33d" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34477 } } peers { permanent_uuid: "e49e56ecdb3840db8078b818692ee573" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45521 } } peers { permanent_uuid: "835c484325134201ad6ecad0451f5d1d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 43253 } }
I20260807 20:16:41.172748 21187 ts_tablet_manager.cc:1428] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 20:16:41.172837 20951 heartbeater.cc:503] Master 127.23.200.254:40127 was elected leader, sending a full tablet report...
I20260807 20:16:41.209019 21192 raft_consensus.cc:482] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:16:41.209174 21192 raft_consensus.cc:504] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41dd4182b050455e919a2207ba8ee33d" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34477 } } peers { permanent_uuid: "e49e56ecdb3840db8078b818692ee573" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45521 } } peers { permanent_uuid: "835c484325134201ad6ecad0451f5d1d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 43253 } }
I20260807 20:16:41.209704 21192 leader_election.cc:288] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41dd4182b050455e919a2207ba8ee33d (127.23.200.195:34477), 835c484325134201ad6ecad0451f5d1d (127.23.200.194:43253)
I20260807 20:16:41.213655 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:41.209901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:43253 (local address 127.23.200.193:54947)
0807 20:16:41.210060 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:41.210071 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:41.210140 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:41.210293 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:41.210294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:41.210296 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:41.210406 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:41.210409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.211338 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.211339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:41.212609 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:41.212611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.213392 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.213394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:41.213468 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:41.213472 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:16:41.213522 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 20:16:41.213567 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":28,"thread_start_us":91,"threads_started":1}
I20260807 20:16:41.213644 21181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:41.209980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:54947 (local address 127.23.200.194:43253)
0807 20:16:41.210075 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:41.210077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:41.210145 (+ 68us) server_negotiation.cc:407] Connection header received
0807 20:16:41.210201 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:41.210202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:41.210212 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:41.210273 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:41.210480 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.211261 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.212688 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.213260 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.213340 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:41.213352 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:41.213535 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 20:16:41.213577 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:16:41.214032 20880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8557adadfdb14c06920433ab8c4c0ca5" candidate_uuid: "e49e56ecdb3840db8078b818692ee573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "835c484325134201ad6ecad0451f5d1d" is_pre_election: true
I20260807 20:16:41.214162 20746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:41.209881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:34477 (local address 127.23.200.193:37751)
0807 20:16:41.209952 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:41.209963 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:41.210004 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:41.210294 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:41.210294 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:41.210296 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:41.210432 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:41.210433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.211420 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.211421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:41.212605 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:41.212609 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.213898 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.213899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:41.213974 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:41.213978 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:16:41.214029 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 20:16:41.214099 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":45}
I20260807 20:16:41.214215 20880 raft_consensus.cc:2454] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e49e56ecdb3840db8078b818692ee573 in term 0.
I20260807 20:16:41.214370 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:41.210076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:37751 (local address 127.23.200.195:34477)
0807 20:16:41.210170 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:41.210172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:41.210179 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:16:41.210202 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:41.210203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:41.210212 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:41.210280 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:41.210494 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.211253 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.213161 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.213683 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.213793 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:41.213804 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:41.214254 (+ 450us) server_negotiation.cc:299] Negotiation successful
0807 20:16:41.214305 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 20:16:41.214483 20563 leader_election.cc:302] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 835c484325134201ad6ecad0451f5d1d, e49e56ecdb3840db8078b818692ee573; no voters:
I20260807 20:16:41.214846 21192 raft_consensus.cc:2781] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:16:41.214846 21083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8557adadfdb14c06920433ab8c4c0ca5" candidate_uuid: "e49e56ecdb3840db8078b818692ee573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41dd4182b050455e919a2207ba8ee33d" is_pre_election: true
I20260807 20:16:41.214924 21192 raft_consensus.cc:482] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:16:41.214949 21192 raft_consensus.cc:3037] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:41.214988 21083 raft_consensus.cc:2454] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e49e56ecdb3840db8078b818692ee573 in term 0.
I20260807 20:16:41.215871 21192 raft_consensus.cc:504] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41dd4182b050455e919a2207ba8ee33d" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34477 } } peers { permanent_uuid: "e49e56ecdb3840db8078b818692ee573" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45521 } } peers { permanent_uuid: "835c484325134201ad6ecad0451f5d1d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 43253 } }
I20260807 20:16:41.216022 21192 leader_election.cc:288] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [CANDIDATE]: Term 1 election: Requested vote from peers 41dd4182b050455e919a2207ba8ee33d (127.23.200.195:34477), 835c484325134201ad6ecad0451f5d1d (127.23.200.194:43253)
I20260807 20:16:41.216176 20880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8557adadfdb14c06920433ab8c4c0ca5" candidate_uuid: "e49e56ecdb3840db8078b818692ee573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "835c484325134201ad6ecad0451f5d1d"
I20260807 20:16:41.216181 21083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8557adadfdb14c06920433ab8c4c0ca5" candidate_uuid: "e49e56ecdb3840db8078b818692ee573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41dd4182b050455e919a2207ba8ee33d"
I20260807 20:16:41.216256 21083 raft_consensus.cc:3037] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:41.216260 20880 raft_consensus.cc:3037] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:41.217140 20880 raft_consensus.cc:2454] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e49e56ecdb3840db8078b818692ee573 in term 1.
I20260807 20:16:41.217154 21083 raft_consensus.cc:2454] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e49e56ecdb3840db8078b818692ee573 in term 1.
I20260807 20:16:41.217304 20563 leader_election.cc:302] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 835c484325134201ad6ecad0451f5d1d, e49e56ecdb3840db8078b818692ee573; no voters:
I20260807 20:16:41.217386 21192 raft_consensus.cc:2781] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:41.217530 21192 raft_consensus.cc:686] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [term 1 LEADER]: Becoming Leader. State: Replica: e49e56ecdb3840db8078b818692ee573, State: Running, Role: LEADER
I20260807 20:16:41.217620 21192 consensus_queue.cc:237] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41dd4182b050455e919a2207ba8ee33d" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34477 } } peers { permanent_uuid: "e49e56ecdb3840db8078b818692ee573" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45521 } } peers { permanent_uuid: "835c484325134201ad6ecad0451f5d1d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 43253 } }
I20260807 20:16:41.218551 20434 catalog_manager.cc:5495] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 reported cstate change: term changed from 0 to 1, leader changed from <none> to e49e56ecdb3840db8078b818692ee573 (127.23.200.193). New cstate: current_term: 1 leader_uuid: "e49e56ecdb3840db8078b818692ee573" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41dd4182b050455e919a2207ba8ee33d" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e49e56ecdb3840db8078b818692ee573" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "835c484325134201ad6ecad0451f5d1d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 43253 } health_report { overall_health: UNKNOWN } } }
I20260807 20:16:41.236196 21179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:41.234104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58532 (local address 127.23.200.193:45521)
0807 20:16:41.234154 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:41.234156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:41.234163 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:16:41.234198 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:41.234199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:41.234219 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:41.234283 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:16:41.234408 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.234859 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.235571 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:41.235751 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:41.235795 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:41.235922 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:16:41.236025 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:16:41.236086 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:41.236098 (+ 12us) server_negotiation.cc:299] Negotiation successful
0807 20:16:41.236116 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 20:16:41.238016 20880 raft_consensus.cc:1261] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d [term 1 FOLLOWER]: Refusing update from remote peer e49e56ecdb3840db8078b818692ee573: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:16:41.238016 21083 raft_consensus.cc:1261] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d [term 1 FOLLOWER]: Refusing update from remote peer e49e56ecdb3840db8078b818692ee573: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:16:41.238368 21192 consensus_queue.cc:1035] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [LEADER]: Connected to new peer: Peer: permanent_uuid: "835c484325134201ad6ecad0451f5d1d" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 43253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:16:41.238518 21200 consensus_queue.cc:1035] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41dd4182b050455e919a2207ba8ee33d" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 34477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:16:41.241012 21205 mvcc.cc:205] Tried to move back new op lower bound from 7316004049868312576 to 7316004049787944960. Current Snapshot: MvccSnapshot[applied={T|T < 7316004049868312576}]
I20260807 20:16:41.241250 21210 mvcc.cc:205] Tried to move back new op lower bound from 7316004049868312576 to 7316004049787944960. Current Snapshot: MvccSnapshot[applied={T|T < 7316004049868312576}]
I20260807 20:16:41.241997 21208 mvcc.cc:205] Tried to move back new op lower bound from 7316004049868312576 to 7316004049787944960. Current Snapshot: MvccSnapshot[applied={T|T < 7316004049868312576}]
I20260807 20:16:41.243605 20432 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:16:41.243705 20432 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:16:41.244477 20432 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5311c8367f14040be44bd21bc2bcfe7: Sending DeleteTablet for 3 replicas of tablet 8557adadfdb14c06920433ab8c4c0ca5
I20260807 20:16:41.244812 20849 tablet_service.cc:1514] Processing DeleteTablet for tablet 8557adadfdb14c06920433ab8c4c0ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:41 UTC) from {username='slave'} at 127.0.0.1:53084
I20260807 20:16:41.244817 21049 tablet_service.cc:1514] Processing DeleteTablet for tablet 8557adadfdb14c06920433ab8c4c0ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:41 UTC) from {username='slave'} at 127.0.0.1:39426
I20260807 20:16:41.244853 20638 tablet_service.cc:1514] Processing DeleteTablet for tablet 8557adadfdb14c06920433ab8c4c0ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:41 UTC) from {username='slave'} at 127.0.0.1:58518
I20260807 20:16:41.245070 21230 tablet_replica.cc:330] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573: stopping tablet replica
I20260807 20:16:41.245198 21230 raft_consensus.cc:2229] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:16:41.245208 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 20532
I20260807 20:16:41.245321 21230 raft_consensus.cc:2258] T 8557adadfdb14c06920433ab8c4c0ca5 P e49e56ecdb3840db8078b818692ee573 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:41.245748 21229 tablet_replica.cc:330] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d: stopping tablet replica
I20260807 20:16:41.245841 21229 raft_consensus.cc:2229] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:16:41.245920 21229 raft_consensus.cc:2258] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:41.248054 21227 tablet_replica.cc:330] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d: stopping tablet replica
I20260807 20:16:41.248164 21227 raft_consensus.cc:2229] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:16:41.248268 21227 raft_consensus.cc:2258] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:41.248615 21229 ts_tablet_manager.cc:1905] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:16:41.248682 21227 ts_tablet_manager.cc:1905] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:16:41.250452 21227 ts_tablet_manager.cc:1918] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:16:41.250456 21229 ts_tablet_manager.cc:1918] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:16:41.250531 21227 log.cc:1196] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/8557adadfdb14c06920433ab8c4c0ca5
I20260807 20:16:41.250531 21229 log.cc:1196] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/8557adadfdb14c06920433ab8c4c0ca5
I20260807 20:16:41.250775 21227 ts_tablet_manager.cc:1939] T 8557adadfdb14c06920433ab8c4c0ca5 P 41dd4182b050455e919a2207ba8ee33d: Deleting consensus metadata
I20260807 20:16:41.250775 21229 ts_tablet_manager.cc:1939] T 8557adadfdb14c06920433ab8c4c0ca5 P 835c484325134201ad6ecad0451f5d1d: Deleting consensus metadata
I20260807 20:16:41.251377 20406 catalog_manager.cc:4841] TS 835c484325134201ad6ecad0451f5d1d (127.23.200.194:43253): tablet 8557adadfdb14c06920433ab8c4c0ca5 (table test-table [id=2ce90c0e4c35478da5fd3d2c18036a04]) successfully deleted
I20260807 20:16:41.253681 20407 catalog_manager.cc:4841] TS 41dd4182b050455e919a2207ba8ee33d (127.23.200.195:34477): tablet 8557adadfdb14c06920433ab8c4c0ca5 (table test-table [id=2ce90c0e4c35478da5fd3d2c18036a04]) successfully deleted
W20260807 20:16:41.258062 20408 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:45521 (error 108)
I20260807 20:16:41.258257 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 20748
I20260807 20:16:41.278405 21178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:41.277364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:45521 (local address 127.0.0.1:58546)
0807 20:16:41.278257 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:41.278346 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:45521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":855}
W20260807 20:16:41.279603 20408 catalog_manager.cc:4584] TS e49e56ecdb3840db8078b818692ee573 (127.23.200.193:45521): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8557adadfdb14c06920433ab8c4c0ca5: Network error: Client connection negotiation failed: client connection to 127.23.200.193:45521: connect: Connection refused (error 111)
I20260807 20:16:41.283557 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 20956
I20260807 20:16:41.297835 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 20367
2026-08-07T20:16:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (753 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:16:41 dist-test-slave-2j7r krb5kdc[21237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:16:41 dist-test-slave-2j7r krb5kdc[21237](info): set up 2 sockets
Aug 07 20:16:41 dist-test-slave-2j7r krb5kdc[21237](info): commencing operation
krb5kdc: starting...
W20260807 20:16:42.451668 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:16:42 dist-test-slave-2j7r krb5kdc[21237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:16:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:16:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:16:42.578305 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:35159
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:35159
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:42.665665 21255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:42.665876 21255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:42.665899 21255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:42.667094 21255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:16:42.667135 21255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:42.667148 21255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:42.667160 21255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:16:42.667171 21255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:16:42.668689 21255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:35159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:35159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:42.668937 21255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:42.669207 21255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:42.674216 21264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:42.674276 21255 server_base.cc:1034] running on GCE node
W20260807 20:16:42.674237 21262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:42.674237 21268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:42.674700 21255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:42.675015 21255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:42.676162 21255 hybrid_clock.cc:648] HybridClock initialized: now 1786133802676148 us; error 31 us; skew 500 ppm
Aug 07 20:16:42 dist-test-slave-2j7r krb5kdc[21237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133802, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:42.678802 21255 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:16:42.679244 21255 webserver.cc:460] Webserver started at http://127.23.200.254:46677/ using document root <none> and password file <none>
I20260807 20:16:42.679476 21255 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:42.679520 21255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:42.679644 21255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:42.680770 21255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "42a9abc33944468f8add27d9b38e15ea"
format_stamp: "Formatted at 2026-08-07 20:16:42 on dist-test-slave-2j7r"
server_key: "b164c44220c858cb9a61852b973de04e"
server_key_iv: "cf260161131d3feafb074cf12be12130"
server_key_version: "encryptionkey@0"
I20260807 20:16:42.681162 21255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "42a9abc33944468f8add27d9b38e15ea"
format_stamp: "Formatted at 2026-08-07 20:16:42 on dist-test-slave-2j7r"
server_key: "b164c44220c858cb9a61852b973de04e"
server_key_iv: "cf260161131d3feafb074cf12be12130"
server_key_version: "encryptionkey@0"
I20260807 20:16:42.682444 21255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:42.683308 21276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:42.683549 21255 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:42.683658 21255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "42a9abc33944468f8add27d9b38e15ea"
format_stamp: "Formatted at 2026-08-07 20:16:42 on dist-test-slave-2j7r"
server_key: "b164c44220c858cb9a61852b973de04e"
server_key_iv: "cf260161131d3feafb074cf12be12130"
server_key_version: "encryptionkey@0"
I20260807 20:16:42.683750 21255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:42.692752 21255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:42.694603 21255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:42.694736 21255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:42.698691 21255 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:35159
I20260807 20:16:42.698745 21365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:35159 every 8 connection(s)
I20260807 20:16:42.699087 21255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:16:42.699944 21366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:42.702270 21366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea: Bootstrap starting.
I20260807 20:16:42.702826 21366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:42.703091 21366 log.cc:824] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:42.703828 21366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea: No bootstrap required, opened a new log
I20260807 20:16:42.703895 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 21255
I20260807 20:16:42.704015 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:16:42.704201 24355 external_mini_cluster.cc:1428] Setting key 9b4eee680ae272e1b04baf01bd17ca64
I20260807 20:16:42.705782 21366 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a9abc33944468f8add27d9b38e15ea" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 35159 } }
I20260807 20:16:42.705920 21366 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:42.705964 21366 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a9abc33944468f8add27d9b38e15ea, State: Initialized, Role: FOLLOWER
I20260807 20:16:42.706113 21366 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a9abc33944468f8add27d9b38e15ea" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 35159 } }
I20260807 20:16:42.706254 21366 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:16:42.706344 21366 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:16:42.706405 21366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:42.707333 21366 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a9abc33944468f8add27d9b38e15ea" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 35159 } }
I20260807 20:16:42.707436 21366 leader_election.cc:302] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a9abc33944468f8add27d9b38e15ea; no voters:
I20260807 20:16:42.707585 21366 leader_election.cc:288] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:16:42.707619 21371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:42.707743 21371 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [term 1 LEADER]: Becoming Leader. State: Replica: 42a9abc33944468f8add27d9b38e15ea, State: Running, Role: LEADER
I20260807 20:16:42.707828 21371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a9abc33944468f8add27d9b38e15ea" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 35159 } }
I20260807 20:16:42.707862 21366 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:16:42.708200 21372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42a9abc33944468f8add27d9b38e15ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a9abc33944468f8add27d9b38e15ea" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 35159 } } }
I20260807 20:16:42.708278 21372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:42.708197 21374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42a9abc33944468f8add27d9b38e15ea. Latest consensus state: current_term: 1 leader_uuid: "42a9abc33944468f8add27d9b38e15ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a9abc33944468f8add27d9b38e15ea" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 35159 } } }
I20260807 20:16:42.708474 21374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:42.708575 21377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:16:42.709066 21377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:16:42.711041 21377 catalog_manager.cc:1347] Generated new cluster ID: 6e623f2d4aa8410d8a9a3ed8c6ff3c4e
I20260807 20:16:42.711091 21377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:16:42 dist-test-slave-2j7r krb5kdc[21237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:16:42.715342 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:42.704783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52842 (local address 127.23.200.254:35159)
0807 20:16:42.704978 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:42.704988 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:42.705018 (+ 30us) server_negotiation.cc:407] Connection header received
0807 20:16:42.705759 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:42.705768 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:42.705828 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:42.705982 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:42.706744 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:42.707780 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:42.709171 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:42.709708 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:42.709818 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:42.712219 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:42.712223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:42.712224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:42.712236 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:42.713771 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:42.714126 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:42.714129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:42.714130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:42.714248 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:42.714488 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:42.714489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:42.714489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:42.714726 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:42.714878 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:42.714994 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 20:16:42.715131 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260807 20:16:42.755615 21377 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:16:42.756191 21377 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:16:42.759250 21377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea: Generated new TSK 0
I20260807 20:16:42.759479 21377 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:16:42.795977 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:35159
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:42.882323 21400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:42.882511 21400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:42.882534 21400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:42.883639 21400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:42.883680 21400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:42.883704 21400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:16:42.885149 21400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:35159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:42.885360 21400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:42.885574 21400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:42.890522 21412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:42.890522 21413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:42.890671 21415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:42.890723 21400 server_base.cc:1034] running on GCE node
I20260807 20:16:42.890956 21400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:42.891177 21400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:42.892344 21400 hybrid_clock.cc:648] HybridClock initialized: now 1786133802892324 us; error 41 us; skew 500 ppm
Aug 07 20:16:42 dist-test-slave-2j7r krb5kdc[21237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133802, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:42.894977 21400 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:16:42.895416 21400 webserver.cc:460] Webserver started at http://127.23.200.193:36177/ using document root <none> and password file <none>
I20260807 20:16:42.895639 21400 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:42.895684 21400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:42.895785 21400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:42.896785 21400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "2b00fe946839416fb020c137fe4fcbc4"
format_stamp: "Formatted at 2026-08-07 20:16:42 on dist-test-slave-2j7r"
server_key: "45e6509d7e813062f289d39ab810cb2c"
server_key_iv: "8a2904eaa4b42aea217a6dda365988cb"
server_key_version: "encryptionkey@0"
I20260807 20:16:42.897150 21400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "2b00fe946839416fb020c137fe4fcbc4"
format_stamp: "Formatted at 2026-08-07 20:16:42 on dist-test-slave-2j7r"
server_key: "45e6509d7e813062f289d39ab810cb2c"
server_key_iv: "8a2904eaa4b42aea217a6dda365988cb"
server_key_version: "encryptionkey@0"
I20260807 20:16:42.898416 21400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:42.899142 21423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:42.899333 21400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:16:42.899410 21400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "2b00fe946839416fb020c137fe4fcbc4"
format_stamp: "Formatted at 2026-08-07 20:16:42 on dist-test-slave-2j7r"
server_key: "45e6509d7e813062f289d39ab810cb2c"
server_key_iv: "8a2904eaa4b42aea217a6dda365988cb"
server_key_version: "encryptionkey@0"
I20260807 20:16:42.899473 21400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:42.911257 21400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:42.912930 21400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:42.913064 21400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:42.913301 21400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:42.913596 21400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:42.913630 21400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:42.913657 21400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:42.913681 21400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:42.919100 21400 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:43433
I20260807 20:16:42.919159 21616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:43433 every 8 connection(s)
I20260807 20:16:42.919445 21400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:16:42.921763 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 21400
I20260807 20:16:42.921883 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:16:42.922096 24355 external_mini_cluster.cc:1428] Setting key 6fcc7ab754ab1a48d8a3f9b0923ae106
Aug 07 20:16:42 dist-test-slave-2j7r krb5kdc[21237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133802, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:16:42.929805 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:42.920142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:44215 (local address 127.23.200.254:35159)
0807 20:16:42.920235 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:42.920238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:42.920425 (+ 187us) server_negotiation.cc:407] Connection header received
0807 20:16:42.920936 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:42.920938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:42.920949 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:42.921003 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:42.922211 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:42.922944 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:42.924235 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:42.924462 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:42.924534 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:42.926840 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:42.926841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:42.926841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:42.926850 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:42.928376 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:42.928784 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:42.928785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:42.928786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:42.928852 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:42.929109 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:42.929110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:42.929110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:42.929257 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:42.929328 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:42.929604 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 20:16:42.929697 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 20:16:42.929965 21621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:42.920113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:35159 (local address 127.23.200.193:44215)
0807 20:16:42.920279 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:42.920338 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:42.920833 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:42.921061 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:42.921064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:42.921758 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:42.922059 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:42.922066 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:42.923037 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:42.923038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:42.924142 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:42.924145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:42.924572 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:42.924573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:42.924733 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:42.925238 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:16:42.925247 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:42.926737 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:42.928467 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:42.928475 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:42.928478 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:42.928704 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:42.928921 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:42.928921 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:42.928921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:42.929030 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:42.929332 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:42.929338 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:42.929475 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 20:16:42.929562 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 20:16:42.929693 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260807 20:16:42.930455 21618 heartbeater.cc:347] Connected to a master server at 127.23.200.254:35159
I20260807 20:16:42.930572 21618 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:42.930799 21618 heartbeater.cc:511] Master 127.23.200.254:35159 requested a full tablet report, sending...
I20260807 20:16:42.931437 21304 ts_manager.cc:194] Registered new tserver with Master: 2b00fe946839416fb020c137fe4fcbc4 (127.23.200.193:43433)
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
I20260807 20:16:42.932384 21304 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:44215
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:16:42.950518 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:35159
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:43.033028 21626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:43.033244 21626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:43.033267 21626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:43.034345 21626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:43.034379 21626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:43.034402 21626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:16:43.035815 21626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:35159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:43.035995 21626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:43.036182 21626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:16:43.041177 21626 server_base.cc:1034] running on GCE node
W20260807 20:16:43.041141 21634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:43.041136 21639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:43.041105 21633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:43.041538 21626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:43.041793 21626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:43.042944 21626 hybrid_clock.cc:648] HybridClock initialized: now 1786133803042929 us; error 36 us; skew 500 ppm
Aug 07 20:16:43 dist-test-slave-2j7r krb5kdc[21237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133803, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:43.045387 21626 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:16:43.045756 21626 webserver.cc:460] Webserver started at http://127.23.200.194:32937/ using document root <none> and password file <none>
I20260807 20:16:43.045965 21626 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:43.046006 21626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:43.046098 21626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:43.047089 21626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "226b2647b73740c289982375e7890634"
format_stamp: "Formatted at 2026-08-07 20:16:43 on dist-test-slave-2j7r"
server_key: "d429827f141146bb5ad8a280af700c2b"
server_key_iv: "91044cef29758e23bc174e15192db54f"
server_key_version: "encryptionkey@0"
I20260807 20:16:43.047374 21626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "226b2647b73740c289982375e7890634"
format_stamp: "Formatted at 2026-08-07 20:16:43 on dist-test-slave-2j7r"
server_key: "d429827f141146bb5ad8a280af700c2b"
server_key_iv: "91044cef29758e23bc174e15192db54f"
server_key_version: "encryptionkey@0"
I20260807 20:16:43.048532 21626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:16:43.049225 21655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:43.049368 21626 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:16:43.049460 21626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "226b2647b73740c289982375e7890634"
format_stamp: "Formatted at 2026-08-07 20:16:43 on dist-test-slave-2j7r"
server_key: "d429827f141146bb5ad8a280af700c2b"
server_key_iv: "91044cef29758e23bc174e15192db54f"
server_key_version: "encryptionkey@0"
I20260807 20:16:43.049516 21626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:43.075304 21626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:43.077548 21626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:43.077669 21626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:43.077880 21626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:43.078171 21626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:43.078202 21626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:43.078223 21626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:43.078239 21626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:43.083720 21626 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:45085
I20260807 20:16:43.083791 21831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:45085 every 8 connection(s)
I20260807 20:16:43.084100 21626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:16:43.085943 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 21626
I20260807 20:16:43.086078 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:16:43.086299 24355 external_mini_cluster.cc:1428] Setting key fe03a8553e3b6c9170f288aa855a2601
Aug 07 20:16:43 dist-test-slave-2j7r krb5kdc[21237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133803, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:16:43.094063 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.084822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:49857 (local address 127.23.200.254:35159)
0807 20:16:43.084910 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:43.084913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:43.085078 (+ 165us) server_negotiation.cc:407] Connection header received
0807 20:16:43.085603 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:43.085604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:43.085616 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:43.085676 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:43.086991 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.087531 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.088849 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.089180 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.089250 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.091209 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:43.091211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:43.091211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:43.091220 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:43.092717 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:43.093122 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:43.093123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:43.093124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:43.093193 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:43.093447 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:43.093447 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:43.093448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:43.093583 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:43.093654 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:43.093871 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 20:16:43.093973 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 20:16:43.094190 21836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.084771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:35159 (local address 127.23.200.194:49857)
0807 20:16:43.084951 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:43.084993 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:43.085507 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:43.085745 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:43.085750 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:43.086593 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:43.086875 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.086882 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.087622 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.087624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.088746 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.088749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.089252 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.089253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.089361 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.089803 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:16:43.089811 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:43.091116 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:43.092803 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:43.092809 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:43.092811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:43.093018 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:43.093255 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:43.093255 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:43.093255 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:43.093385 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:43.093643 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:43.093647 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:43.093759 (+ 112us) client_negotiation.cc:769] Sending connection context
0807 20:16:43.093834 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 20:16:43.093962 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":105,"threads_started":1}
I20260807 20:16:43.094645 21832 heartbeater.cc:347] Connected to a master server at 127.23.200.254:35159
I20260807 20:16:43.094733 21832 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:43.094911 21832 heartbeater.cc:511] Master 127.23.200.254:35159 requested a full tablet report, sending...
I20260807 20:16:43.095294 21304 ts_manager.cc:194] Registered new tserver with Master: 226b2647b73740c289982375e7890634 (127.23.200.194:45085)
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
I20260807 20:16:43.095666 21304 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:49857
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:16:43.114113 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:35159
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:43.197948 21839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:43.198160 21839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:43.198192 21839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:43.199331 21839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:43.199373 21839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:43.199406 21839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:16:43.200861 21839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:35159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:43.201116 21839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:43.201347 21839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:43.206365 21846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:43.206470 21851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:43.206339 21847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:43.206612 21839 server_base.cc:1034] running on GCE node
I20260807 20:16:43.206897 21839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:43.207145 21839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:43.208290 21839 hybrid_clock.cc:648] HybridClock initialized: now 1786133803208268 us; error 34 us; skew 500 ppm
Aug 07 20:16:43 dist-test-slave-2j7r krb5kdc[21237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133803, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:43.211011 21839 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:16:43.211483 21839 webserver.cc:460] Webserver started at http://127.23.200.195:42717/ using document root <none> and password file <none>
I20260807 20:16:43.211687 21839 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:43.211730 21839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:43.211853 21839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:43.212926 21839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "2a09fda9ae0649e88b9bf3215eb082ae"
format_stamp: "Formatted at 2026-08-07 20:16:43 on dist-test-slave-2j7r"
server_key: "1bf2ab1a2763669a004a9fc5b51523cf"
server_key_iv: "ac6edfc03fbb2263dfbc1e9962683bb9"
server_key_version: "encryptionkey@0"
I20260807 20:16:43.213292 21839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "2a09fda9ae0649e88b9bf3215eb082ae"
format_stamp: "Formatted at 2026-08-07 20:16:43 on dist-test-slave-2j7r"
server_key: "1bf2ab1a2763669a004a9fc5b51523cf"
server_key_iv: "ac6edfc03fbb2263dfbc1e9962683bb9"
server_key_version: "encryptionkey@0"
I20260807 20:16:43.214560 21839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:43.215235 21860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:43.215461 21839 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:16:43.215551 21839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "2a09fda9ae0649e88b9bf3215eb082ae"
format_stamp: "Formatted at 2026-08-07 20:16:43 on dist-test-slave-2j7r"
server_key: "1bf2ab1a2763669a004a9fc5b51523cf"
server_key_iv: "ac6edfc03fbb2263dfbc1e9962683bb9"
server_key_version: "encryptionkey@0"
I20260807 20:16:43.215620 21839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:43.225824 21839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:43.227936 21839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:43.228057 21839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:43.228296 21839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:43.228605 21839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:43.228639 21839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:43.228667 21839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:43.228693 21839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:43.234174 21839 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:37957
I20260807 20:16:43.234236 22036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:37957 every 8 connection(s)
I20260807 20:16:43.234537 21839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:16:43.239709 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 21839
I20260807 20:16:43.239811 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:16:43.240011 24355 external_mini_cluster.cc:1428] Setting key 31d881300d494cb02a60b5ef9f3f09e5
Aug 07 20:16:43 dist-test-slave-2j7r krb5kdc[21237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133803, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:16:43.243765 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.235343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:36035 (local address 127.23.200.254:35159)
0807 20:16:43.235405 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:43.235407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:43.235552 (+ 145us) server_negotiation.cc:407] Connection header received
0807 20:16:43.236023 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:43.236024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:43.236033 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:43.236091 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:43.237026 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.237566 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.238622 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.238813 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.238866 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.241417 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:43.241418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:43.241419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:43.241426 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:43.242686 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:43.243072 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:43.243073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:43.243073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:43.243119 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:43.243296 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:43.243297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:43.243297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:43.243401 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:43.243445 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:43.243644 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 20:16:43.243702 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:16:43.243966 22040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.235261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:35159 (local address 127.23.200.195:36035)
0807 20:16:43.235428 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:43.235472 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:43.235948 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:43.236147 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:43.236151 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:43.236730 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:43.236935 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.236940 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.237662 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.237663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.238526 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.238530 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.238884 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.238885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.239006 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.239511 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:16:43.239522 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:43.241324 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:43.242764 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:43.242770 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:43.242772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:43.243011 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:43.243155 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:43.243156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:43.243156 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:43.243257 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:43.243443 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:43.243446 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:43.243557 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 20:16:43.243633 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 20:16:43.243749 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260807 20:16:43.244419 22037 heartbeater.cc:347] Connected to a master server at 127.23.200.254:35159
I20260807 20:16:43.244521 22037 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:43.244695 22037 heartbeater.cc:511] Master 127.23.200.254:35159 requested a full tablet report, sending...
I20260807 20:16:43.245044 21304 ts_manager.cc:194] Registered new tserver with Master: 2a09fda9ae0649e88b9bf3215eb082ae (127.23.200.195:37957)
I20260807 20:16:43.245411 21304 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:36035
I20260807 20:16:43.251854 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:16:43.258991 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.253254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52854 (local address 127.23.200.254:35159)
0807 20:16:43.253300 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:43.253302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:43.253373 (+ 71us) server_negotiation.cc:407] Connection header received
0807 20:16:43.253483 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:43.253484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:43.253496 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:43.253536 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:43.254061 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.254530 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.255219 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.255408 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.255461 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.256730 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:43.256732 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:43.256732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:43.256740 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:43.257834 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:43.258224 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:43.258225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:43.258225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:43.258278 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:43.258471 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:43.258472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:43.258472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:43.258604 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:43.258697 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:43.258829 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 20:16:43.258897 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:16:43.260594 21304 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:16:43.262256 21304 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:16:43.268504 22052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.264495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:37957 (local address 127.0.0.1:52818)
0807 20:16:43.264614 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:43.264657 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:43.264723 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:43.265300 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:43.265301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:43.265303 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:43.265340 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.265342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.266560 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.266561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.267398 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.267400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.268240 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.268241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.268312 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.268322 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 20:16:43.268374 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 20:16:43.268424 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":65,"threads_started":1}
I20260807 20:16:43.268509 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.264823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52818 (local address 127.23.200.195:37957)
0807 20:16:43.265010 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:43.265015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:43.265026 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:16:43.265065 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:43.265067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:43.265105 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:43.265188 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:43.265420 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.266488 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.267478 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.268160 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.268229 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.268296 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:43.268389 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 20:16:43.268432 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
I20260807 20:16:43.268923 22053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.264495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:45085 (local address 127.0.0.1:32954)
0807 20:16:43.264657 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:43.264666 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:43.264722 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:43.265054 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:43.265057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:43.265059 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:43.265158 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.265160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.266482 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.266483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.267550 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.267554 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.268412 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.268413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.268467 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.268470 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:16:43.268518 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 20:16:43.268856 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":5,"thread_start_us":65,"threads_started":1}
I20260807 20:16:43.269174 22055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.264613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32954 (local address 127.23.200.194:45085)
0807 20:16:43.264885 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:43.264891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:43.264901 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:16:43.264942 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:43.264944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:43.264961 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:43.265038 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:43.265243 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.266382 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.267641 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.268329 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.268395 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.268442 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:43.269032 (+ 590us) server_negotiation.cc:299] Negotiation successful
0807 20:16:43.269078 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":105,"threads_started":1}
I20260807 20:16:43.269336 21933 tablet_service.cc:1467] Processing CreateTablet for tablet 100b94988dea4e4597ec68a80ae8349d (DEFAULT_TABLE table=test-table [id=fdd6300e58814e83b2154b376b89e7c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:43.269726 21933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 100b94988dea4e4597ec68a80ae8349d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:43.269905 22056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.264692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:43433 (local address 127.0.0.1:43616)
0807 20:16:43.264908 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:43.264917 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:43.264959 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:43.265412 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:43.265413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:43.265416 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:43.265460 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.265461 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.266304 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.266305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.267565 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.267567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.269614 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.269615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.269737 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.269740 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:16:43.269797 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 20:16:43.269841 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":2,"thread_start_us":70,"threads_started":1}
I20260807 20:16:43.269809 21727 tablet_service.cc:1467] Processing CreateTablet for tablet 100b94988dea4e4597ec68a80ae8349d (DEFAULT_TABLE table=test-table [id=fdd6300e58814e83b2154b376b89e7c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:43.270123 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.264930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43616 (local address 127.23.200.193:43433)
0807 20:16:43.265189 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:43.265193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:43.265202 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:16:43.265238 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:43.265240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:43.265254 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:43.265326 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:43.265535 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.266210 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.268477 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.269361 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.269426 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.269504 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:43.270003 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 20:16:43.270049 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":85,"threads_started":1}
I20260807 20:16:43.270130 21727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 100b94988dea4e4597ec68a80ae8349d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:43.270759 21512 tablet_service.cc:1467] Processing CreateTablet for tablet 100b94988dea4e4597ec68a80ae8349d (DEFAULT_TABLE table=test-table [id=fdd6300e58814e83b2154b376b89e7c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:43.271092 21512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 100b94988dea4e4597ec68a80ae8349d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:43.272693 22060 tablet_bootstrap.cc:492] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae: Bootstrap starting.
I20260807 20:16:43.273248 22061 tablet_bootstrap.cc:492] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634: Bootstrap starting.
I20260807 20:16:43.273337 22060 tablet_bootstrap.cc:654] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:43.273876 22060 log.cc:824] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:43.273904 22061 tablet_bootstrap.cc:654] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:43.274220 22066 tablet_bootstrap.cc:492] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4: Bootstrap starting.
I20260807 20:16:43.274443 22061 log.cc:824] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:43.274784 22060 tablet_bootstrap.cc:492] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae: No bootstrap required, opened a new log
I20260807 20:16:43.274837 22066 tablet_bootstrap.cc:654] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:43.274844 22060 ts_tablet_manager.cc:1397] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:16:43.275187 22061 tablet_bootstrap.cc:492] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634: No bootstrap required, opened a new log
I20260807 20:16:43.275254 22061 ts_tablet_manager.cc:1397] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:16:43.275295 22066 log.cc:824] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:43.275897 22066 tablet_bootstrap.cc:492] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4: No bootstrap required, opened a new log
I20260807 20:16:43.275954 22066 ts_tablet_manager.cc:1397] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:16:43.276738 22060 raft_consensus.cc:348] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a09fda9ae0649e88b9bf3215eb082ae" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37957 } } peers { permanent_uuid: "2b00fe946839416fb020c137fe4fcbc4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 43433 } } peers { permanent_uuid: "226b2647b73740c289982375e7890634" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45085 } }
I20260807 20:16:43.276922 22060 raft_consensus.cc:374] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:43.276958 22060 raft_consensus.cc:729] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a09fda9ae0649e88b9bf3215eb082ae, State: Initialized, Role: FOLLOWER
I20260807 20:16:43.277139 22060 consensus_queue.cc:260] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a09fda9ae0649e88b9bf3215eb082ae" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37957 } } peers { permanent_uuid: "2b00fe946839416fb020c137fe4fcbc4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 43433 } } peers { permanent_uuid: "226b2647b73740c289982375e7890634" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45085 } }
I20260807 20:16:43.277456 22037 heartbeater.cc:503] Master 127.23.200.254:35159 was elected leader, sending a full tablet report...
I20260807 20:16:43.277366 22066 raft_consensus.cc:348] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a09fda9ae0649e88b9bf3215eb082ae" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37957 } } peers { permanent_uuid: "2b00fe946839416fb020c137fe4fcbc4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 43433 } } peers { permanent_uuid: "226b2647b73740c289982375e7890634" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45085 } }
I20260807 20:16:43.277520 22066 raft_consensus.cc:374] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:43.277560 22066 raft_consensus.cc:729] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b00fe946839416fb020c137fe4fcbc4, State: Initialized, Role: FOLLOWER
I20260807 20:16:43.277519 22061 raft_consensus.cc:348] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a09fda9ae0649e88b9bf3215eb082ae" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37957 } } peers { permanent_uuid: "2b00fe946839416fb020c137fe4fcbc4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 43433 } } peers { permanent_uuid: "226b2647b73740c289982375e7890634" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45085 } }
I20260807 20:16:43.277679 22061 raft_consensus.cc:374] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:43.277669 22066 consensus_queue.cc:260] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a09fda9ae0649e88b9bf3215eb082ae" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37957 } } peers { permanent_uuid: "2b00fe946839416fb020c137fe4fcbc4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 43433 } } peers { permanent_uuid: "226b2647b73740c289982375e7890634" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45085 } }
I20260807 20:16:43.277724 22061 raft_consensus.cc:729] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 226b2647b73740c289982375e7890634, State: Initialized, Role: FOLLOWER
I20260807 20:16:43.277927 22066 ts_tablet_manager.cc:1428] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:16:43.277870 22061 consensus_queue.cc:260] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a09fda9ae0649e88b9bf3215eb082ae" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37957 } } peers { permanent_uuid: "2b00fe946839416fb020c137fe4fcbc4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 43433 } } peers { permanent_uuid: "226b2647b73740c289982375e7890634" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45085 } }
I20260807 20:16:43.278034 21618 heartbeater.cc:503] Master 127.23.200.254:35159 was elected leader, sending a full tablet report...
I20260807 20:16:43.277454 22060 ts_tablet_manager.cc:1428] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:16:43.278168 22061 ts_tablet_manager.cc:1428] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:16:43.278270 21832 heartbeater.cc:503] Master 127.23.200.254:35159 was elected leader, sending a full tablet report...
W20260807 20:16:43.334748 21833 tablet.cc:2369] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:16:43.355198 22071 raft_consensus.cc:482] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:16:43.355321 22071 raft_consensus.cc:504] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a09fda9ae0649e88b9bf3215eb082ae" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37957 } } peers { permanent_uuid: "2b00fe946839416fb020c137fe4fcbc4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 43433 } } peers { permanent_uuid: "226b2647b73740c289982375e7890634" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45085 } }
I20260807 20:16:43.355713 22071 leader_election.cc:288] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a09fda9ae0649e88b9bf3215eb082ae (127.23.200.195:37957), 226b2647b73740c289982375e7890634 (127.23.200.194:45085)
I20260807 20:16:43.359511 22055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.356094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:46059 (local address 127.23.200.194:45085)
0807 20:16:43.356138 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:43.356140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:43.356179 (+ 39us) server_negotiation.cc:407] Connection header received
0807 20:16:43.356254 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:43.356255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:43.356263 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:43.356314 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:43.356440 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.357147 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.358573 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.359165 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.359235 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.359243 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:43.359399 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 20:16:43.359442 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:16:43.359511 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.356018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:32861 (local address 127.23.200.195:37957)
0807 20:16:43.356072 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:43.356074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:43.356083 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:16:43.356111 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:43.356111 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:43.356122 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:43.356171 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:43.356373 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.357136 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.358573 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.359157 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.359234 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.359243 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:43.359400 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 20:16:43.359443 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 20:16:43.359517 21621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.355884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:37957 (local address 127.23.200.193:32861)
0807 20:16:43.355969 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:43.355982 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:43.356032 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:43.356186 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:43.356187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:43.356189 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:43.356259 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.356261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.357215 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.357216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.358474 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.358477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.359241 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.359242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.359324 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.359329 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:16:43.359420 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 20:16:43.359449 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":31}
I20260807 20:16:43.359524 22073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.355900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:45085 (local address 127.23.200.193:46059)
0807 20:16:43.356130 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:43.356139 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:43.356200 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:43.356327 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:43.356328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:43.356330 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:43.356385 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.356387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.357207 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.357207 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.358474 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.358477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.359241 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.359242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.359323 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.359330 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 20:16:43.359385 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 20:16:43.359431 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":30,"thread_start_us":114,"threads_started":1}
I20260807 20:16:43.359930 21963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "100b94988dea4e4597ec68a80ae8349d" candidate_uuid: "2b00fe946839416fb020c137fe4fcbc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a09fda9ae0649e88b9bf3215eb082ae" is_pre_election: true
I20260807 20:16:43.360097 21963 raft_consensus.cc:2454] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b00fe946839416fb020c137fe4fcbc4 in term 0.
I20260807 20:16:43.360086 21756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "100b94988dea4e4597ec68a80ae8349d" candidate_uuid: "2b00fe946839416fb020c137fe4fcbc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "226b2647b73740c289982375e7890634" is_pre_election: true
I20260807 20:16:43.360209 21756 raft_consensus.cc:2454] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b00fe946839416fb020c137fe4fcbc4 in term 0.
I20260807 20:16:43.360375 21426 leader_election.cc:302] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a09fda9ae0649e88b9bf3215eb082ae, 2b00fe946839416fb020c137fe4fcbc4; no voters:
I20260807 20:16:43.360550 22071 raft_consensus.cc:2781] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:16:43.360589 22071 raft_consensus.cc:482] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:16:43.360610 22071 raft_consensus.cc:3037] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:43.361343 22071 raft_consensus.cc:504] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a09fda9ae0649e88b9bf3215eb082ae" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37957 } } peers { permanent_uuid: "2b00fe946839416fb020c137fe4fcbc4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 43433 } } peers { permanent_uuid: "226b2647b73740c289982375e7890634" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45085 } }
I20260807 20:16:43.361538 22071 leader_election.cc:288] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [CANDIDATE]: Term 1 election: Requested vote from peers 2a09fda9ae0649e88b9bf3215eb082ae (127.23.200.195:37957), 226b2647b73740c289982375e7890634 (127.23.200.194:45085)
I20260807 20:16:43.361629 21756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "100b94988dea4e4597ec68a80ae8349d" candidate_uuid: "2b00fe946839416fb020c137fe4fcbc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "226b2647b73740c289982375e7890634"
I20260807 20:16:43.361678 21963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "100b94988dea4e4597ec68a80ae8349d" candidate_uuid: "2b00fe946839416fb020c137fe4fcbc4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a09fda9ae0649e88b9bf3215eb082ae"
I20260807 20:16:43.361709 21756 raft_consensus.cc:3037] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:43.361749 21963 raft_consensus.cc:3037] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:43.362587 21963 raft_consensus.cc:2454] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b00fe946839416fb020c137fe4fcbc4 in term 1.
I20260807 20:16:43.362587 21756 raft_consensus.cc:2454] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b00fe946839416fb020c137fe4fcbc4 in term 1.
I20260807 20:16:43.362746 21429 leader_election.cc:302] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 226b2647b73740c289982375e7890634, 2b00fe946839416fb020c137fe4fcbc4; no voters:
I20260807 20:16:43.362823 22071 raft_consensus.cc:2781] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:43.362974 22071 raft_consensus.cc:686] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [term 1 LEADER]: Becoming Leader. State: Replica: 2b00fe946839416fb020c137fe4fcbc4, State: Running, Role: LEADER
I20260807 20:16:43.363066 22071 consensus_queue.cc:237] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a09fda9ae0649e88b9bf3215eb082ae" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37957 } } peers { permanent_uuid: "2b00fe946839416fb020c137fe4fcbc4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 43433 } } peers { permanent_uuid: "226b2647b73740c289982375e7890634" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45085 } }
I20260807 20:16:43.363917 21301 catalog_manager.cc:5495] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b00fe946839416fb020c137fe4fcbc4 (127.23.200.193). New cstate: current_term: 1 leader_uuid: "2b00fe946839416fb020c137fe4fcbc4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a09fda9ae0649e88b9bf3215eb082ae" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b00fe946839416fb020c137fe4fcbc4" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 43433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "226b2647b73740c289982375e7890634" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45085 } health_report { overall_health: UNKNOWN } } }
I20260807 20:16:43.383484 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.381403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43630 (local address 127.23.200.193:43433)
0807 20:16:43.381458 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:43.381460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:43.381468 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:16:43.381502 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:43.381503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:43.381521 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:43.381584 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:16:43.381690 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.382165 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.382881 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.383065 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.383115 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.383216 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:16:43.383305 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:16:43.383376 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:43.383415 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 20:16:43.383424 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 20:16:43.385293 21756 raft_consensus.cc:1261] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634 [term 1 FOLLOWER]: Refusing update from remote peer 2b00fe946839416fb020c137fe4fcbc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:16:43.385331 21963 raft_consensus.cc:1261] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae [term 1 FOLLOWER]: Refusing update from remote peer 2b00fe946839416fb020c137fe4fcbc4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:16:43.385644 22074 consensus_queue.cc:1035] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "226b2647b73740c289982375e7890634" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 45085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:16:43.385771 22071 consensus_queue.cc:1035] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a09fda9ae0649e88b9bf3215eb082ae" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:16:43.388727 22079 mvcc.cc:205] Tried to move back new op lower bound from 7316004058663317504 to 7316004058575675392. Current Snapshot: MvccSnapshot[applied={T|T < 7316004058663317504}]
I20260807 20:16:43.389441 22081 mvcc.cc:205] Tried to move back new op lower bound from 7316004058663317504 to 7316004058575675392. Current Snapshot: MvccSnapshot[applied={T|T < 7316004058663317504}]
I20260807 20:16:43.390830 21301 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:16:43.390921 21301 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:16:43.391616 21301 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42a9abc33944468f8add27d9b38e15ea: Sending DeleteTablet for 3 replicas of tablet 100b94988dea4e4597ec68a80ae8349d
I20260807 20:16:43.391963 21727 tablet_service.cc:1514] Processing DeleteTablet for tablet 100b94988dea4e4597ec68a80ae8349d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:43 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:32954
I20260807 20:16:43.391989 21933 tablet_service.cc:1514] Processing DeleteTablet for tablet 100b94988dea4e4597ec68a80ae8349d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:43 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:52818
I20260807 20:16:43.392203 22093 tablet_replica.cc:330] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634: stopping tablet replica
I20260807 20:16:43.392313 22093 raft_consensus.cc:2229] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:16:43.392330 21512 tablet_service.cc:1514] Processing DeleteTablet for tablet 100b94988dea4e4597ec68a80ae8349d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:43 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:43616
I20260807 20:16:43.392427 22093 raft_consensus.cc:2258] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:43.392590 22094 tablet_replica.cc:330] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae: stopping tablet replica
I20260807 20:16:43.392678 22095 tablet_replica.cc:330] T 100b94988dea4e4597ec68a80ae8349d P 2b00fe946839416fb020c137fe4fcbc4: stopping tablet replica
I20260807 20:16:43.392761 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 21400
I20260807 20:16:43.418483 22093 ts_tablet_manager.cc:1905] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:16:43.392680 22094 raft_consensus.cc:2229] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:16:43.419843 22094 raft_consensus.cc:2258] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:43.420413 22094 ts_tablet_manager.cc:1905] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:16:43.420806 22093 ts_tablet_manager.cc:1918] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:16:43.420899 22093 log.cc:1196] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/100b94988dea4e4597ec68a80ae8349d
I20260807 20:16:43.421231 22093 ts_tablet_manager.cc:1939] T 100b94988dea4e4597ec68a80ae8349d P 226b2647b73740c289982375e7890634: Deleting consensus metadata
I20260807 20:16:43.421763 21281 catalog_manager.cc:4841] TS 226b2647b73740c289982375e7890634 (127.23.200.194:45085): tablet 100b94988dea4e4597ec68a80ae8349d (table test-table [id=fdd6300e58814e83b2154b376b89e7c6]) successfully deleted
I20260807 20:16:43.421964 22094 ts_tablet_manager.cc:1918] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:16:43.422022 22094 log.cc:1196] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/100b94988dea4e4597ec68a80ae8349d
I20260807 20:16:43.422248 22094 ts_tablet_manager.cc:1939] T 100b94988dea4e4597ec68a80ae8349d P 2a09fda9ae0649e88b9bf3215eb082ae: Deleting consensus metadata
I20260807 20:16:43.422632 21277 catalog_manager.cc:4841] TS 2a09fda9ae0649e88b9bf3215eb082ae (127.23.200.195:37957): tablet 100b94988dea4e4597ec68a80ae8349d (table test-table [id=fdd6300e58814e83b2154b376b89e7c6]) successfully deleted
W20260807 20:16:43.424156 21277 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:43433 (error 108)
I20260807 20:16:43.424474 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 21626
I20260807 20:16:43.424674 22056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.424553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:43433 (local address 127.0.0.1:43646)
0807 20:16:43.424604 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:43.424640 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 20:16:43.429561 21277 catalog_manager.cc:4584] TS 2b00fe946839416fb020c137fe4fcbc4 (127.23.200.193:43433): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 100b94988dea4e4597ec68a80ae8349d: Network error: Client connection negotiation failed: client connection to 127.23.200.193:43433: connect: Connection refused (error 111)
I20260807 20:16:43.434979 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 21839
I20260807 20:16:43.445716 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 21255
2026-08-07T20:16:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2149 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T20:16:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:16:43Z Disabled control of system clock
I20260807 20:16:43.478855 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:38911
--webserver_interface=127.23.200.254
--webserver_port=0
--builtin_ntp_servers=127.23.200.212:37539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:38911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:16:43.560912 22103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:43.561159 22103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:43.561180 22103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:43.562369 22103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:16:43.562412 22103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:43.562424 22103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:43.562435 22103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:16:43.562454 22103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:16:43.563858 22103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:37539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:38911
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:38911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:43.564095 22103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:43.564390 22103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:43.569209 22112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:43.569263 22113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:43.569339 22103 server_base.cc:1034] running on GCE node
W20260807 20:16:43.569353 22118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:43.569732 22103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:43.570020 22103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:43.571169 22103 hybrid_clock.cc:648] HybridClock initialized: now 1786133803571148 us; error 33 us; skew 500 ppm
I20260807 20:16:43.576227 22103 webserver.cc:460] Webserver started at http://127.23.200.254:44295/ using document root <none> and password file <none>
I20260807 20:16:43.576504 22103 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:43.576561 22103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:43.576719 22103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:43.577926 22103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "d65c6c0306fb447297c9ed98cd6f1f08"
format_stamp: "Formatted at 2026-08-07 20:16:43 on dist-test-slave-2j7r"
server_key: "b6b6aaa596cf8cf7eaa5577efb1812fe"
server_key_iv: "1486139c98b45699974bb8b3405ca952"
server_key_version: "encryptionkey@0"
I20260807 20:16:43.578289 22103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "d65c6c0306fb447297c9ed98cd6f1f08"
format_stamp: "Formatted at 2026-08-07 20:16:43 on dist-test-slave-2j7r"
server_key: "b6b6aaa596cf8cf7eaa5577efb1812fe"
server_key_iv: "1486139c98b45699974bb8b3405ca952"
server_key_version: "encryptionkey@0"
I20260807 20:16:43.579592 22103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:16:43.580488 22125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:43.580703 22103 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:43.580806 22103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "d65c6c0306fb447297c9ed98cd6f1f08"
format_stamp: "Formatted at 2026-08-07 20:16:43 on dist-test-slave-2j7r"
server_key: "b6b6aaa596cf8cf7eaa5577efb1812fe"
server_key_iv: "1486139c98b45699974bb8b3405ca952"
server_key_version: "encryptionkey@0"
I20260807 20:16:43.580870 22103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:43.593451 22103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:43.593780 22103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:43.593916 22103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:43.597867 22103 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:38911
I20260807 20:16:43.597918 22206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:38911 every 8 connection(s)
I20260807 20:16:43.598255 22103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:16:43.599203 22207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:43.601613 22207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08: Bootstrap starting.
I20260807 20:16:43.602293 22207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:43.602574 22207 log.cc:824] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:43.603210 22207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08: No bootstrap required, opened a new log
I20260807 20:16:43.603391 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 22103
I20260807 20:16:43.603471 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:16:43.603644 24355 external_mini_cluster.cc:1428] Setting key 9c9c808fbce5a6ddc08f7d54d13238d4
I20260807 20:16:43.605367 22207 raft_consensus.cc:348] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65c6c0306fb447297c9ed98cd6f1f08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38911 } }
I20260807 20:16:43.605552 22207 raft_consensus.cc:374] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:43.605589 22207 raft_consensus.cc:729] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d65c6c0306fb447297c9ed98cd6f1f08, State: Initialized, Role: FOLLOWER
I20260807 20:16:43.605747 22207 consensus_queue.cc:260] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65c6c0306fb447297c9ed98cd6f1f08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38911 } }
I20260807 20:16:43.605835 22207 raft_consensus.cc:388] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:16:43.605867 22207 raft_consensus.cc:482] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:16:43.605914 22207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:43.606626 22207 raft_consensus.cc:504] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65c6c0306fb447297c9ed98cd6f1f08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38911 } }
I20260807 20:16:43.606745 22207 leader_election.cc:302] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d65c6c0306fb447297c9ed98cd6f1f08; no voters:
I20260807 20:16:43.606948 22207 leader_election.cc:288] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:16:43.606971 22219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:43.607149 22219 raft_consensus.cc:686] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [term 1 LEADER]: Becoming Leader. State: Replica: d65c6c0306fb447297c9ed98cd6f1f08, State: Running, Role: LEADER
I20260807 20:16:43.607285 22219 consensus_queue.cc:237] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65c6c0306fb447297c9ed98cd6f1f08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38911 } }
I20260807 20:16:43.607331 22207 sys_catalog.cc:564] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:16:43.607638 22219 sys_catalog.cc:455] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d65c6c0306fb447297c9ed98cd6f1f08. Latest consensus state: current_term: 1 leader_uuid: "d65c6c0306fb447297c9ed98cd6f1f08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65c6c0306fb447297c9ed98cd6f1f08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38911 } } }
I20260807 20:16:43.608958 22222 sys_catalog.cc:455] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d65c6c0306fb447297c9ed98cd6f1f08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65c6c0306fb447297c9ed98cd6f1f08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 38911 } } }
I20260807 20:16:43.609072 22222 sys_catalog.cc:458] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:43.608986 22219 sys_catalog.cc:458] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08 [sys.catalog]: This master's current role is: LEADER
W20260807 20:16:43.609230 22243 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:16:43.609285 22243 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:16:43.609372 22245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:16:43.609526 22245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:16:43.610853 22218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.604202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57516 (local address 127.23.200.254:38911)
0807 20:16:43.604427 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:43.604436 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:43.604469 (+ 33us) server_negotiation.cc:407] Connection header received
0807 20:16:43.605325 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:43.605334 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:43.605417 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:43.605568 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:43.605796 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.607618 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.608610 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.609339 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.609451 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.610134 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:43.610138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:43.610139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:43.610335 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:43.610378 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:43.610393 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:43.610450 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:43.610564 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 20:16:43.610657 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260807 20:16:43.611043 22245 catalog_manager.cc:1347] Generated new cluster ID: a487786015484085ad928bfbd7fc83fa
I20260807 20:16:43.611109 22245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:16:43.623622 22245 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:16:43.624199 22245 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:16:43.628443 22245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08: Generated new TSK 0
I20260807 20:16:43.628625 22245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:16:43.638916 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:38911
--builtin_ntp_servers=127.23.200.212:37539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:16:43.721912 22252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:43.722119 22252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:43.722150 22252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:43.723286 22252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:43.723325 22252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:43.723357 22252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:16:43.724713 22252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:37539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:38911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:43.724959 22252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:43.725286 22252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:43.730288 22262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:43.730252 22260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:43.730249 22259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:43.731006 22252 server_base.cc:1034] running on GCE node
I20260807 20:16:43.731208 22252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:43.731477 22252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:43.732632 22252 hybrid_clock.cc:648] HybridClock initialized: now 1786133803732620 us; error 28 us; skew 500 ppm
I20260807 20:16:43.733906 22252 webserver.cc:460] Webserver started at http://127.23.200.193:42013/ using document root <none> and password file <none>
I20260807 20:16:43.734119 22252 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:43.734159 22252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:43.734258 22252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:43.735337 22252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "626fdbc4ac1941cc97353f168b243742"
format_stamp: "Formatted at 2026-08-07 20:16:43 on dist-test-slave-2j7r"
server_key: "17ab04d4e8b401d91732e08bc5fbc978"
server_key_iv: "39660aa05a4c9788cd42489a0ee3dd3f"
server_key_version: "encryptionkey@0"
I20260807 20:16:43.735630 22252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "626fdbc4ac1941cc97353f168b243742"
format_stamp: "Formatted at 2026-08-07 20:16:43 on dist-test-slave-2j7r"
server_key: "17ab04d4e8b401d91732e08bc5fbc978"
server_key_iv: "39660aa05a4c9788cd42489a0ee3dd3f"
server_key_version: "encryptionkey@0"
I20260807 20:16:43.736835 22252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:16:43.737655 22271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:43.737826 22252 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:16:43.737921 22252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "626fdbc4ac1941cc97353f168b243742"
format_stamp: "Formatted at 2026-08-07 20:16:43 on dist-test-slave-2j7r"
server_key: "17ab04d4e8b401d91732e08bc5fbc978"
server_key_iv: "39660aa05a4c9788cd42489a0ee3dd3f"
server_key_version: "encryptionkey@0"
I20260807 20:16:43.737979 22252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:43.748879 22252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:43.749161 22252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:43.749266 22252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:43.749444 22252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:43.749692 22252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:43.749720 22252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:43.749747 22252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:43.749763 22252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:43.755544 22252 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:37683
I20260807 20:16:43.755589 22466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:37683 every 8 connection(s)
I20260807 20:16:43.755895 22252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:16:43.760531 22218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.756557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:42125 (local address 127.23.200.254:38911)
0807 20:16:43.756631 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:43.756634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:43.756840 (+ 206us) server_negotiation.cc:407] Connection header received
0807 20:16:43.757314 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:43.757315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:43.757328 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:43.757375 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:43.758005 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.758506 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.759414 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.759593 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.759636 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.760322 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:43.760323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:43.760323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:43.760331 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:43.760345 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:43.760349 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:43.760413 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:43.760450 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 20:16:43.760472 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 20:16:43.760733 22473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.756534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:38911 (local address 127.23.200.193:42125)
0807 20:16:43.756723 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:43.756755 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:43.757236 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:43.757437 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:43.757441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:43.757449 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:43.757946 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.757950 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.758568 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.758568 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.759342 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.759344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.759646 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.759646 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.759747 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.760201 (+ 454us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:16:43.760201 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:43.760229 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:16:43.760251 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:43.760393 (+ 142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:43.760397 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:43.760399 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:16:43.760445 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 20:16:43.760519 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260807 20:16:43.761227 22467 heartbeater.cc:347] Connected to a master server at 127.23.200.254:38911
I20260807 20:16:43.761319 22467 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:43.761471 22467 heartbeater.cc:511] Master 127.23.200.254:38911 requested a full tablet report, sending...
I20260807 20:16:43.761932 22147 ts_manager.cc:194] Registered new tserver with Master: 626fdbc4ac1941cc97353f168b243742 (127.23.200.193:37683)
I20260807 20:16:43.762602 22147 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.200.193:42125
I20260807 20:16:43.765000 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 22252
I20260807 20:16:43.765108 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:16:43.765292 24355 external_mini_cluster.cc:1428] Setting key 3d812efec29e2bf33d18caa1efd1e352
I20260807 20:16:43.766153 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:38911
--builtin_ntp_servers=127.23.200.212:37539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:16:43.848628 22474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:43.848836 22474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:43.848858 22474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:43.850005 22474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:43.850051 22474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:43.850083 22474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:16:43.851430 22474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:37539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:38911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:43.851634 22474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:43.851890 22474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:43.856846 22488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:43.856855 22482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:43.857064 22474 server_base.cc:1034] running on GCE node
W20260807 20:16:43.856894 22483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:43.857358 22474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:43.857592 22474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:43.858736 22474 hybrid_clock.cc:648] HybridClock initialized: now 1786133803858726 us; error 28 us; skew 500 ppm
I20260807 20:16:43.859988 22474 webserver.cc:460] Webserver started at http://127.23.200.194:32821/ using document root <none> and password file <none>
I20260807 20:16:43.860216 22474 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:43.860280 22474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:43.860459 22474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:43.861618 22474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "3e57891b48db400fbfa91429d9aefc7a"
format_stamp: "Formatted at 2026-08-07 20:16:43 on dist-test-slave-2j7r"
server_key: "cc445266cfee7d7334476ef096792444"
server_key_iv: "9389a74555870109c5c571bb8acfa5cf"
server_key_version: "encryptionkey@0"
I20260807 20:16:43.861946 22474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "3e57891b48db400fbfa91429d9aefc7a"
format_stamp: "Formatted at 2026-08-07 20:16:43 on dist-test-slave-2j7r"
server_key: "cc445266cfee7d7334476ef096792444"
server_key_iv: "9389a74555870109c5c571bb8acfa5cf"
server_key_version: "encryptionkey@0"
I20260807 20:16:43.863247 22474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:16:43.864048 22495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:43.864192 22474 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:16:43.864279 22474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "3e57891b48db400fbfa91429d9aefc7a"
format_stamp: "Formatted at 2026-08-07 20:16:43 on dist-test-slave-2j7r"
server_key: "cc445266cfee7d7334476ef096792444"
server_key_iv: "9389a74555870109c5c571bb8acfa5cf"
server_key_version: "encryptionkey@0"
I20260807 20:16:43.864383 22474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:43.910641 22474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:43.910969 22474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:43.911106 22474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:43.911357 22474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:43.911710 22474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:43.911744 22474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:43.911774 22474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:43.911791 22474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:43.917644 22474 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:46447
I20260807 20:16:43.917695 22672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:46447 every 8 connection(s)
I20260807 20:16:43.918033 22474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:16:43.922312 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 22474
I20260807 20:16:43.922427 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:16:43.922646 24355 external_mini_cluster.cc:1428] Setting key e66e784ce5c457591e6d44dabc530e6e
I20260807 20:16:43.923866 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:38911
--builtin_ntp_servers=127.23.200.212:37539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 20:16:43.923909 22218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.918856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:43645 (local address 127.23.200.254:38911)
0807 20:16:43.918923 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:43.918926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:43.919154 (+ 228us) server_negotiation.cc:407] Connection header received
0807 20:16:43.919773 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:43.919774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:43.919789 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:43.919850 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:43.920590 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.921157 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.922256 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.922550 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.922617 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.923649 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:43.923651 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:43.923651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:43.923655 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:43.923671 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:43.923675 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:43.923717 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:43.923811 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 20:16:43.923838 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:16:43.924147 22679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:43.918785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:38911 (local address 127.23.200.194:43645)
0807 20:16:43.919020 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:43.919060 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:43.919644 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:43.919925 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:43.919930 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:43.919942 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:43.920487 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.920492 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.921278 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.921279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.922132 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:43.922136 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:43.922647 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:43.922649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:43.922824 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:43.923478 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:16:43.923478 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:43.923521 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:16:43.923554 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:43.923730 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:43.923735 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:43.923739 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:16:43.923807 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 20:16:43.923926 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":91,"threads_started":1}
I20260807 20:16:43.924618 22676 heartbeater.cc:347] Connected to a master server at 127.23.200.254:38911
I20260807 20:16:43.924723 22676 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:43.924926 22676 heartbeater.cc:511] Master 127.23.200.254:38911 requested a full tablet report, sending...
I20260807 20:16:43.925432 22147 ts_manager.cc:194] Registered new tserver with Master: 3e57891b48db400fbfa91429d9aefc7a (127.23.200.194:46447)
I20260807 20:16:43.925827 22147 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.200.194:43645
W20260807 20:16:44.007421 22682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:44.007632 22682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:44.007652 22682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:44.008898 22682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:44.008940 22682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:44.008965 22682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:16:44.010350 22682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:37539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:38911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:44.010591 22682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:44.010881 22682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:44.015909 22690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:44.016048 22682 server_base.cc:1034] running on GCE node
W20260807 20:16:44.015909 22688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:44.015915 22693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:44.016350 22682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:44.016605 22682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:44.017748 22682 hybrid_clock.cc:648] HybridClock initialized: now 1786133804017726 us; error 31 us; skew 500 ppm
I20260807 20:16:44.019037 22682 webserver.cc:460] Webserver started at http://127.23.200.195:44049/ using document root <none> and password file <none>
I20260807 20:16:44.019292 22682 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:44.019356 22682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:44.019480 22682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:44.020558 22682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "03451ee7292b4f279adb95b51db400bf"
format_stamp: "Formatted at 2026-08-07 20:16:44 on dist-test-slave-2j7r"
server_key: "1ae060eae4e85fe2e87322c249501f21"
server_key_iv: "ff04e6dfd4f3b6e4681cc33872564832"
server_key_version: "encryptionkey@0"
I20260807 20:16:44.020900 22682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "03451ee7292b4f279adb95b51db400bf"
format_stamp: "Formatted at 2026-08-07 20:16:44 on dist-test-slave-2j7r"
server_key: "1ae060eae4e85fe2e87322c249501f21"
server_key_iv: "ff04e6dfd4f3b6e4681cc33872564832"
server_key_version: "encryptionkey@0"
I20260807 20:16:44.022224 22682 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:16:44.023123 22703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:44.023286 22682 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:44.023432 22682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "03451ee7292b4f279adb95b51db400bf"
format_stamp: "Formatted at 2026-08-07 20:16:44 on dist-test-slave-2j7r"
server_key: "1ae060eae4e85fe2e87322c249501f21"
server_key_iv: "ff04e6dfd4f3b6e4681cc33872564832"
server_key_version: "encryptionkey@0"
I20260807 20:16:44.023505 22682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:44.036624 22682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:44.036931 22682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:44.037063 22682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:44.037335 22682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:44.037667 22682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:44.037700 22682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:44.037730 22682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:44.037755 22682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:44.043286 22682 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:37221
I20260807 20:16:44.043340 22895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:37221 every 8 connection(s)
I20260807 20:16:44.043668 22682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:16:44.048573 22218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:44.044414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:52147 (local address 127.23.200.254:38911)
0807 20:16:44.044472 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:44.044474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:44.044706 (+ 232us) server_negotiation.cc:407] Connection header received
0807 20:16:44.045218 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:44.045219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:44.045230 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:44.045285 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:44.045939 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.046459 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.047406 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.047589 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.047636 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:44.048337 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:44.048338 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:44.048339 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:44.048343 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:44.048358 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:44.048362 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:44.048424 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:44.048484 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 20:16:44.048512 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:16:44.048801 22899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:44.044383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:38911 (local address 127.23.200.195:52147)
0807 20:16:44.044585 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:44.044627 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:44.045131 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:44.045364 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:44.045368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:44.045377 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:44.045876 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:44.045881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.046545 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.046546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:44.047339 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:44.047341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.047645 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.047646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:44.047749 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:44.048216 (+ 467us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:16:44.048216 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:44.048243 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:16:44.048264 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:44.048427 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:44.048429 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:44.048432 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:16:44.048476 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 20:16:44.048567 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260807 20:16:44.049340 22896 heartbeater.cc:347] Connected to a master server at 127.23.200.254:38911
I20260807 20:16:44.049424 22896 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:44.049597 22896 heartbeater.cc:511] Master 127.23.200.254:38911 requested a full tablet report, sending...
I20260807 20:16:44.049973 22147 ts_manager.cc:194] Registered new tserver with Master: 03451ee7292b4f279adb95b51db400bf (127.23.200.195:37221)
I20260807 20:16:44.050105 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 22682
I20260807 20:16:44.050190 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:16:44.050371 24355 external_mini_cluster.cc:1428] Setting key 30ca4ac0cec275c8c25908e8637a350b
I20260807 20:16:44.050417 22147 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.200.195:52147
I20260807 20:16:44.051241 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:16:44.055522 22218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:44.052616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57524 (local address 127.23.200.254:38911)
0807 20:16:44.052653 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:44.052654 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:44.052762 (+ 108us) server_negotiation.cc:407] Connection header received
0807 20:16:44.052800 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:44.052801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:44.052812 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:44.052862 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:44.052996 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.053541 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.054354 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.054546 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.054601 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:44.055197 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:44.055198 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:44.055199 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:44.055203 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:44.055214 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:44.055218 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:44.055346 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:44.055454 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 20:16:44.055465 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:16:44.057171 22147 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57524:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:16:44.058756 22147 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:16:44.065044 22918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:44.061254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:37221 (local address 127.0.0.1:54220)
0807 20:16:44.061735 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:44.061744 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:44.061787 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:44.062035 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:44.062036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:44.062038 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:44.062071 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:44.062072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.063231 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.063232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:44.064092 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:44.064095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.064739 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.064740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:44.064817 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:44.064831 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 20:16:44.064904 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 20:16:44.064958 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":3,"thread_start_us":69,"threads_started":1}
I20260807 20:16:44.065047 22928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:44.061351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54220 (local address 127.23.200.195:37221)
0807 20:16:44.061536 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:44.061540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:44.061837 (+ 297us) server_negotiation.cc:407] Connection header received
0807 20:16:44.061877 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:44.061880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:44.061895 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:44.061973 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:44.062150 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.063160 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.064166 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.064663 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.064731 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:44.064823 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:44.064924 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 20:16:44.064961 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260807 20:16:44.065284 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:44.060998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:37683 (local address 127.0.0.1:36472)
0807 20:16:44.061171 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:44.061215 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:44.061285 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:44.061610 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:44.061613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:44.061615 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:44.061747 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:44.061749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.063067 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.063068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:44.064132 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:44.064134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.065053 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.065054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:44.065137 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:44.065141 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:16:44.065172 (+ 31us) client_negotiation.cc:240] Negotiation successful
0807 20:16:44.065211 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":27,"thread_start_us":108,"threads_started":1}
I20260807 20:16:44.065439 22929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:44.061399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46506 (local address 127.23.200.194:46447)
0807 20:16:44.061590 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:44.061595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:44.061606 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:16:44.061685 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:44.061688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:44.061704 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:44.061776 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:44.061981 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.062718 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.064213 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.064950 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.065012 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:44.065123 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:44.065321 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 20:16:44.065362 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":78,"threads_started":1}
I20260807 20:16:44.065598 22923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:44.060998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:46447 (local address 127.0.0.1:46506)
0807 20:16:44.061538 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:44.061549 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:44.061601 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:44.061850 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:44.061850 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:44.061853 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:44.061902 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:44.061904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.062794 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.062795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:44.064106 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:44.064109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.065153 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.065154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:44.065204 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:44.065207 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:16:44.065250 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 20:16:44.065281 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":2,"thread_start_us":108,"threads_started":1}
I20260807 20:16:44.065845 22920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:44.061140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36472 (local address 127.23.200.193:37683)
0807 20:16:44.061416 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:44.061421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:44.061431 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:16:44.061470 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:44.061472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:44.061489 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:44.061600 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:44.061828 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.062963 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.064215 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.064940 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.065600 (+ 660us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:44.065699 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:44.065726 (+ 27us) server_negotiation.cc:299] Negotiation successful
0807 20:16:44.065768 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":108,"threads_started":1}
I20260807 20:16:44.065897 22776 tablet_service.cc:1467] Processing CreateTablet for tablet 37eb9dab17be486dba79ad8b0c750335 (DEFAULT_TABLE table=test-table [id=1a0a65e09d7c44cfa3d6de3dc238597c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:44.066300 22776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37eb9dab17be486dba79ad8b0c750335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:44.066288 22570 tablet_service.cc:1467] Processing CreateTablet for tablet 37eb9dab17be486dba79ad8b0c750335 (DEFAULT_TABLE table=test-table [id=1a0a65e09d7c44cfa3d6de3dc238597c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:44.066636 22570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37eb9dab17be486dba79ad8b0c750335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:44.066535 22358 tablet_service.cc:1467] Processing CreateTablet for tablet 37eb9dab17be486dba79ad8b0c750335 (DEFAULT_TABLE table=test-table [id=1a0a65e09d7c44cfa3d6de3dc238597c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:44.066841 22358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37eb9dab17be486dba79ad8b0c750335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:44.069787 22932 tablet_bootstrap.cc:492] T 37eb9dab17be486dba79ad8b0c750335 P 626fdbc4ac1941cc97353f168b243742: Bootstrap starting.
I20260807 20:16:44.070359 22932 tablet_bootstrap.cc:654] T 37eb9dab17be486dba79ad8b0c750335 P 626fdbc4ac1941cc97353f168b243742: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:44.070583 22932 log.cc:824] T 37eb9dab17be486dba79ad8b0c750335 P 626fdbc4ac1941cc97353f168b243742: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:44.070852 22933 tablet_bootstrap.cc:492] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf: Bootstrap starting.
I20260807 20:16:44.071252 22932 tablet_bootstrap.cc:492] T 37eb9dab17be486dba79ad8b0c750335 P 626fdbc4ac1941cc97353f168b243742: No bootstrap required, opened a new log
I20260807 20:16:44.071318 22932 ts_tablet_manager.cc:1397] T 37eb9dab17be486dba79ad8b0c750335 P 626fdbc4ac1941cc97353f168b243742: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:16:44.071375 22933 tablet_bootstrap.cc:654] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:44.071623 22933 log.cc:824] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:44.071925 22934 tablet_bootstrap.cc:492] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a: Bootstrap starting.
I20260807 20:16:44.072393 22933 tablet_bootstrap.cc:492] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf: No bootstrap required, opened a new log
I20260807 20:16:44.072510 22933 ts_tablet_manager.cc:1397] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:16:44.072536 22934 tablet_bootstrap.cc:654] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:44.072896 22934 log.cc:824] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:44.073510 22934 tablet_bootstrap.cc:492] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a: No bootstrap required, opened a new log
I20260807 20:16:44.073566 22934 ts_tablet_manager.cc:1397] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:16:44.073732 22932 raft_consensus.cc:348] T 37eb9dab17be486dba79ad8b0c750335 P 626fdbc4ac1941cc97353f168b243742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e57891b48db400fbfa91429d9aefc7a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46447 } } peers { permanent_uuid: "03451ee7292b4f279adb95b51db400bf" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37221 } } peers { permanent_uuid: "626fdbc4ac1941cc97353f168b243742" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37683 } }
I20260807 20:16:44.073909 22932 raft_consensus.cc:374] T 37eb9dab17be486dba79ad8b0c750335 P 626fdbc4ac1941cc97353f168b243742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:44.073956 22932 raft_consensus.cc:729] T 37eb9dab17be486dba79ad8b0c750335 P 626fdbc4ac1941cc97353f168b243742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 626fdbc4ac1941cc97353f168b243742, State: Initialized, Role: FOLLOWER
I20260807 20:16:44.074097 22932 consensus_queue.cc:260] T 37eb9dab17be486dba79ad8b0c750335 P 626fdbc4ac1941cc97353f168b243742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e57891b48db400fbfa91429d9aefc7a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46447 } } peers { permanent_uuid: "03451ee7292b4f279adb95b51db400bf" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37221 } } peers { permanent_uuid: "626fdbc4ac1941cc97353f168b243742" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37683 } }
I20260807 20:16:44.074389 22932 ts_tablet_manager.cc:1428] T 37eb9dab17be486dba79ad8b0c750335 P 626fdbc4ac1941cc97353f168b243742: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:16:44.074419 22467 heartbeater.cc:503] Master 127.23.200.254:38911 was elected leader, sending a full tablet report...
I20260807 20:16:44.075106 22933 raft_consensus.cc:348] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e57891b48db400fbfa91429d9aefc7a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46447 } } peers { permanent_uuid: "03451ee7292b4f279adb95b51db400bf" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37221 } } peers { permanent_uuid: "626fdbc4ac1941cc97353f168b243742" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37683 } }
I20260807 20:16:44.075260 22933 raft_consensus.cc:374] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:44.075295 22933 raft_consensus.cc:729] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03451ee7292b4f279adb95b51db400bf, State: Initialized, Role: FOLLOWER
I20260807 20:16:44.075402 22933 consensus_queue.cc:260] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e57891b48db400fbfa91429d9aefc7a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46447 } } peers { permanent_uuid: "03451ee7292b4f279adb95b51db400bf" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37221 } } peers { permanent_uuid: "626fdbc4ac1941cc97353f168b243742" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37683 } }
I20260807 20:16:44.075645 22933 ts_tablet_manager.cc:1428] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 20:16:44.075793 22896 heartbeater.cc:503] Master 127.23.200.254:38911 was elected leader, sending a full tablet report...
I20260807 20:16:44.075686 22934 raft_consensus.cc:348] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e57891b48db400fbfa91429d9aefc7a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46447 } } peers { permanent_uuid: "03451ee7292b4f279adb95b51db400bf" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37221 } } peers { permanent_uuid: "626fdbc4ac1941cc97353f168b243742" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37683 } }
I20260807 20:16:44.075835 22934 raft_consensus.cc:374] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:44.075863 22934 raft_consensus.cc:729] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e57891b48db400fbfa91429d9aefc7a, State: Initialized, Role: FOLLOWER
I20260807 20:16:44.075970 22934 consensus_queue.cc:260] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e57891b48db400fbfa91429d9aefc7a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46447 } } peers { permanent_uuid: "03451ee7292b4f279adb95b51db400bf" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37221 } } peers { permanent_uuid: "626fdbc4ac1941cc97353f168b243742" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37683 } }
I20260807 20:16:44.076188 22934 ts_tablet_manager.cc:1428] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:16:44.076232 22676 heartbeater.cc:503] Master 127.23.200.254:38911 was elected leader, sending a full tablet report...
W20260807 20:16:44.168602 22677 tablet.cc:2369] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:16:44.185515 22943 raft_consensus.cc:482] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:16:44.185644 22943 raft_consensus.cc:504] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e57891b48db400fbfa91429d9aefc7a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46447 } } peers { permanent_uuid: "03451ee7292b4f279adb95b51db400bf" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37221 } } peers { permanent_uuid: "626fdbc4ac1941cc97353f168b243742" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37683 } }
I20260807 20:16:44.186049 22943 leader_election.cc:288] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03451ee7292b4f279adb95b51db400bf (127.23.200.195:37221), 626fdbc4ac1941cc97353f168b243742 (127.23.200.193:37683)
I20260807 20:16:44.189850 22920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:44.186392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:50891 (local address 127.23.200.193:37683)
0807 20:16:44.186463 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:44.186465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:44.186475 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:16:44.186548 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:44.186549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:44.186558 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:44.186599 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:44.186743 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.187438 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.188909 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.189497 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.189573 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:44.189582 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:44.189753 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 20:16:44.189788 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 20:16:44.189850 22928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:44.186283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:34283 (local address 127.23.200.195:37221)
0807 20:16:44.186342 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:44.186345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:44.186354 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:16:44.186417 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:44.186418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:44.186427 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:44.186484 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:44.186651 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.187453 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.188918 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.189497 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.189576 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:44.189582 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:44.189734 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 20:16:44.189775 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 20:16:44.189858 22679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:44.186221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:37221 (local address 127.23.200.194:34283)
0807 20:16:44.186286 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:44.186298 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:44.186343 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:44.186507 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:44.186508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:44.186510 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:44.186579 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:44.186581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.187536 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.187537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:44.188811 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:44.188815 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.189575 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.189576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:44.189660 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:44.189664 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:16:44.189722 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 20:16:44.189777 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":31}
I20260807 20:16:44.189865 22944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:44.186236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:37683 (local address 127.23.200.194:50891)
0807 20:16:44.186423 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:44.186433 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:44.186487 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:44.186623 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:44.186624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:44.186625 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:44.186684 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:44.186685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.187536 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.187537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:44.188813 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:44.188815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.189575 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.189576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:44.189660 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:44.189664 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:16:44.189722 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 20:16:44.189793 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":76,"thread_start_us":102,"threads_started":1}
I20260807 20:16:44.190224 22806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37eb9dab17be486dba79ad8b0c750335" candidate_uuid: "3e57891b48db400fbfa91429d9aefc7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03451ee7292b4f279adb95b51db400bf" is_pre_election: true
I20260807 20:16:44.190237 22387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37eb9dab17be486dba79ad8b0c750335" candidate_uuid: "3e57891b48db400fbfa91429d9aefc7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "626fdbc4ac1941cc97353f168b243742" is_pre_election: true
I20260807 20:16:44.190388 22806 raft_consensus.cc:2454] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e57891b48db400fbfa91429d9aefc7a in term 0.
I20260807 20:16:44.190389 22387 raft_consensus.cc:2454] T 37eb9dab17be486dba79ad8b0c750335 P 626fdbc4ac1941cc97353f168b243742 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e57891b48db400fbfa91429d9aefc7a in term 0.
I20260807 20:16:44.190619 22499 leader_election.cc:302] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03451ee7292b4f279adb95b51db400bf, 3e57891b48db400fbfa91429d9aefc7a; no voters:
I20260807 20:16:44.190797 22943 raft_consensus.cc:2781] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:16:44.190835 22943 raft_consensus.cc:482] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:16:44.190850 22943 raft_consensus.cc:3037] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:44.191591 22943 raft_consensus.cc:504] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e57891b48db400fbfa91429d9aefc7a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46447 } } peers { permanent_uuid: "03451ee7292b4f279adb95b51db400bf" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37221 } } peers { permanent_uuid: "626fdbc4ac1941cc97353f168b243742" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37683 } }
I20260807 20:16:44.191704 22943 leader_election.cc:288] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [CANDIDATE]: Term 1 election: Requested vote from peers 03451ee7292b4f279adb95b51db400bf (127.23.200.195:37221), 626fdbc4ac1941cc97353f168b243742 (127.23.200.193:37683)
I20260807 20:16:44.191887 22806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37eb9dab17be486dba79ad8b0c750335" candidate_uuid: "3e57891b48db400fbfa91429d9aefc7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03451ee7292b4f279adb95b51db400bf"
I20260807 20:16:44.191920 22387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37eb9dab17be486dba79ad8b0c750335" candidate_uuid: "3e57891b48db400fbfa91429d9aefc7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "626fdbc4ac1941cc97353f168b243742"
I20260807 20:16:44.191952 22806 raft_consensus.cc:3037] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:44.192003 22387 raft_consensus.cc:3037] T 37eb9dab17be486dba79ad8b0c750335 P 626fdbc4ac1941cc97353f168b243742 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:44.192627 22806 raft_consensus.cc:2454] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e57891b48db400fbfa91429d9aefc7a in term 1.
I20260807 20:16:44.192772 22387 raft_consensus.cc:2454] T 37eb9dab17be486dba79ad8b0c750335 P 626fdbc4ac1941cc97353f168b243742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e57891b48db400fbfa91429d9aefc7a in term 1.
I20260807 20:16:44.192790 22499 leader_election.cc:302] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03451ee7292b4f279adb95b51db400bf, 3e57891b48db400fbfa91429d9aefc7a; no voters:
I20260807 20:16:44.192996 22943 raft_consensus.cc:2781] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:44.193171 22943 raft_consensus.cc:686] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [term 1 LEADER]: Becoming Leader. State: Replica: 3e57891b48db400fbfa91429d9aefc7a, State: Running, Role: LEADER
I20260807 20:16:44.193265 22943 consensus_queue.cc:237] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e57891b48db400fbfa91429d9aefc7a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46447 } } peers { permanent_uuid: "03451ee7292b4f279adb95b51db400bf" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37221 } } peers { permanent_uuid: "626fdbc4ac1941cc97353f168b243742" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37683 } }
I20260807 20:16:44.194154 22146 catalog_manager.cc:5495] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e57891b48db400fbfa91429d9aefc7a (127.23.200.194). New cstate: current_term: 1 leader_uuid: "3e57891b48db400fbfa91429d9aefc7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e57891b48db400fbfa91429d9aefc7a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 46447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03451ee7292b4f279adb95b51db400bf" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "626fdbc4ac1941cc97353f168b243742" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37683 } health_report { overall_health: UNKNOWN } } }
I20260807 20:16:44.210289 22929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:44.207808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46514 (local address 127.23.200.194:46447)
0807 20:16:44.207870 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:44.207872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:44.207882 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:16:44.207936 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:44.207937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:44.207962 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:44.208023 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:16:44.208142 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.208741 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.209503 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:44.209708 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:44.209768 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:44.209918 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:16:44.210022 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:16:44.210113 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:44.210161 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 20:16:44.210171 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:16:44.212246 22387 raft_consensus.cc:1261] T 37eb9dab17be486dba79ad8b0c750335 P 626fdbc4ac1941cc97353f168b243742 [term 1 FOLLOWER]: Refusing update from remote peer 3e57891b48db400fbfa91429d9aefc7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:16:44.212317 22806 raft_consensus.cc:1261] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf [term 1 FOLLOWER]: Refusing update from remote peer 3e57891b48db400fbfa91429d9aefc7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:16:44.212694 22943 consensus_queue.cc:1035] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "03451ee7292b4f279adb95b51db400bf" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 37221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:16:44.212850 22946 consensus_queue.cc:1035] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "626fdbc4ac1941cc97353f168b243742" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 37683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:16:44.216140 22955 mvcc.cc:205] Tried to move back new op lower bound from 7316004062050480128 to 7316004061976338432. Current Snapshot: MvccSnapshot[applied={T|T < 7316004062050480128 or (T in {7316004062050480128})}]
I20260807 20:16:44.216907 22952 mvcc.cc:205] Tried to move back new op lower bound from 7316004062050480128 to 7316004061976338432. Current Snapshot: MvccSnapshot[applied={T|T < 7316004062050480128 or (T in {7316004062050480128})}]
I20260807 20:16:44.217990 22146 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:16:44.218099 22146 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:16:44.218922 22146 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d65c6c0306fb447297c9ed98cd6f1f08: Sending DeleteTablet for 3 replicas of tablet 37eb9dab17be486dba79ad8b0c750335
I20260807 20:16:44.219305 22358 tablet_service.cc:1514] Processing DeleteTablet for tablet 37eb9dab17be486dba79ad8b0c750335 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:44 UTC) from {username='slave'} at 127.0.0.1:36472
I20260807 20:16:44.219307 22570 tablet_service.cc:1514] Processing DeleteTablet for tablet 37eb9dab17be486dba79ad8b0c750335 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:44 UTC) from {username='slave'} at 127.0.0.1:46506
I20260807 20:16:44.219313 22776 tablet_service.cc:1514] Processing DeleteTablet for tablet 37eb9dab17be486dba79ad8b0c750335 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:44 UTC) from {username='slave'} at 127.0.0.1:54220
I20260807 20:16:44.219555 22970 tablet_replica.cc:330] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf: stopping tablet replica
I20260807 20:16:44.219655 22970 raft_consensus.cc:2229] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:16:44.219760 22970 raft_consensus.cc:2258] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:44.219930 22968 tablet_replica.cc:330] T 37eb9dab17be486dba79ad8b0c750335 P 626fdbc4ac1941cc97353f168b243742: stopping tablet replica
I20260807 20:16:44.219945 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 22252
I20260807 20:16:44.228963 22969 tablet_replica.cc:330] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a: stopping tablet replica
I20260807 20:16:44.229149 22969 raft_consensus.cc:2229] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:16:44.229277 22969 raft_consensus.cc:2258] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:44.234935 22970 ts_tablet_manager.cc:1905] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:16:44.236825 22970 ts_tablet_manager.cc:1918] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:16:44.236936 22970 log.cc:1196] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/37eb9dab17be486dba79ad8b0c750335
I20260807 20:16:44.237303 22970 ts_tablet_manager.cc:1939] T 37eb9dab17be486dba79ad8b0c750335 P 03451ee7292b4f279adb95b51db400bf: Deleting consensus metadata
I20260807 20:16:44.237880 22130 catalog_manager.cc:4841] TS 03451ee7292b4f279adb95b51db400bf (127.23.200.195:37221): tablet 37eb9dab17be486dba79ad8b0c750335 (table test-table [id=1a0a65e09d7c44cfa3d6de3dc238597c]) successfully deleted
I20260807 20:16:44.251988 22969 ts_tablet_manager.cc:1905] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:16:44.254506 22969 ts_tablet_manager.cc:1918] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:16:44.254591 22969 log.cc:1196] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/37eb9dab17be486dba79ad8b0c750335
I20260807 20:16:44.254807 22969 ts_tablet_manager.cc:1939] T 37eb9dab17be486dba79ad8b0c750335 P 3e57891b48db400fbfa91429d9aefc7a: Deleting consensus metadata
I20260807 20:16:44.255221 22130 catalog_manager.cc:4841] TS 3e57891b48db400fbfa91429d9aefc7a (127.23.200.194:46447): tablet 37eb9dab17be486dba79ad8b0c750335 (table test-table [id=1a0a65e09d7c44cfa3d6de3dc238597c]) successfully deleted
W20260807 20:16:44.260780 22131 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:37683 (error 108)
I20260807 20:16:44.260948 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 22474
I20260807 20:16:44.261819 22923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:44.261279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:37683 (local address 127.0.0.1:36480)
0807 20:16:44.261721 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:44.261777 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:37683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415}
W20260807 20:16:44.263569 22131 catalog_manager.cc:4584] TS 626fdbc4ac1941cc97353f168b243742 (127.23.200.193:37683): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 37eb9dab17be486dba79ad8b0c750335: Network error: Client connection negotiation failed: client connection to 127.23.200.193:37683: connect: Connection refused (error 111)
I20260807 20:16:44.272310 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 22682
I20260807 20:16:44.287487 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 22103
2026-08-07T20:16:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (841 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:16:44 dist-test-slave-2j7r krb5kdc[22976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:16:44 dist-test-slave-2j7r krb5kdc[22976](info): set up 2 sockets
Aug 07 20:16:44 dist-test-slave-2j7r krb5kdc[22976](info): commencing operation
krb5kdc: starting...
W20260807 20:16:45.447894 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:16:45 dist-test-slave-2j7r krb5kdc[22976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:16:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:16:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:16:45.570096 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:45551
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:44451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:45551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:45.652343 22998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:45.652539 22998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:45.652558 22998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:45.653690 22998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:16:45.653729 22998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:45.653750 22998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:45.653761 22998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:16:45.653774 22998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:16:45.655508 22998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:45551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:45551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:45.655762 22998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:45.655992 22998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:45.660995 23007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:45.660995 23010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:45.661024 23005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:45.661140 22998 server_base.cc:1034] running on GCE node
I20260807 20:16:45.661420 22998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:45.661727 22998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:45.662919 22998 hybrid_clock.cc:648] HybridClock initialized: now 1786133805662873 us; error 41 us; skew 500 ppm
Aug 07 20:16:45 dist-test-slave-2j7r krb5kdc[22976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133805, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:45.665978 22998 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:16:45.666522 22998 webserver.cc:460] Webserver started at http://127.23.200.254:39055/ using document root <none> and password file <none>
I20260807 20:16:45.666782 22998 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:45.666843 22998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:45.666980 22998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:45.668334 22998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "9ca65b5eae4b461ebf3edf1c40fd622a"
format_stamp: "Formatted at 2026-08-07 20:16:45 on dist-test-slave-2j7r"
server_key: "cac91f01447c3b966959ead3cdb7d1a1"
server_key_iv: "43c71cef540651718dea18df1a0d6b6a"
server_key_version: "encryptionkey@0"
I20260807 20:16:45.668694 22998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "9ca65b5eae4b461ebf3edf1c40fd622a"
format_stamp: "Formatted at 2026-08-07 20:16:45 on dist-test-slave-2j7r"
server_key: "cac91f01447c3b966959ead3cdb7d1a1"
server_key_iv: "43c71cef540651718dea18df1a0d6b6a"
server_key_version: "encryptionkey@0"
I20260807 20:16:45.670488 22998 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260807 20:16:45.672451 23022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:45.672616 22998 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 20:16:45.672711 22998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "9ca65b5eae4b461ebf3edf1c40fd622a"
format_stamp: "Formatted at 2026-08-07 20:16:45 on dist-test-slave-2j7r"
server_key: "cac91f01447c3b966959ead3cdb7d1a1"
server_key_iv: "43c71cef540651718dea18df1a0d6b6a"
server_key_version: "encryptionkey@0"
I20260807 20:16:45.672788 22998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:45.703291 22998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:45.705282 22998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:45.705428 22998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:45.709398 22998 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:45551
I20260807 20:16:45.709444 23108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:45551 every 8 connection(s)
I20260807 20:16:45.709791 22998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:16:45.710667 23109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:45.713003 23109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a: Bootstrap starting.
I20260807 20:16:45.713665 23109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:45.713938 23109 log.cc:824] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:45.714603 23109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a: No bootstrap required, opened a new log
I20260807 20:16:45.715564 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 22998
I20260807 20:16:45.715693 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:16:45.715875 24355 external_mini_cluster.cc:1428] Setting key e0e3352b6e5611bc4373c0f9e79dfb8b
I20260807 20:16:45.716574 23109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca65b5eae4b461ebf3edf1c40fd622a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45551 } }
I20260807 20:16:45.716715 23109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:45.716737 23109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ca65b5eae4b461ebf3edf1c40fd622a, State: Initialized, Role: FOLLOWER
I20260807 20:16:45.716849 23109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca65b5eae4b461ebf3edf1c40fd622a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45551 } }
I20260807 20:16:45.716928 23109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:16:45.717003 23109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:16:45.717058 23109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:45.717746 23109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca65b5eae4b461ebf3edf1c40fd622a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45551 } }
I20260807 20:16:45.717836 23109 leader_election.cc:302] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ca65b5eae4b461ebf3edf1c40fd622a; no voters:
I20260807 20:16:45.717991 23109 leader_election.cc:288] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:16:45.718032 23117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:45.718183 23117 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [term 1 LEADER]: Becoming Leader. State: Replica: 9ca65b5eae4b461ebf3edf1c40fd622a, State: Running, Role: LEADER
I20260807 20:16:45.718250 23109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:16:45.718353 23117 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca65b5eae4b461ebf3edf1c40fd622a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45551 } }
I20260807 20:16:45.718742 23117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ca65b5eae4b461ebf3edf1c40fd622a. Latest consensus state: current_term: 1 leader_uuid: "9ca65b5eae4b461ebf3edf1c40fd622a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca65b5eae4b461ebf3edf1c40fd622a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45551 } } }
I20260807 20:16:45.719008 23117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:45.718918 23119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ca65b5eae4b461ebf3edf1c40fd622a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca65b5eae4b461ebf3edf1c40fd622a" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 45551 } } }
I20260807 20:16:45.719420 23119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:45.719568 23131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:16:45.720248 23131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:16:45.721679 23131 catalog_manager.cc:1347] Generated new cluster ID: cd50e27f05194813b6ccb4fb431d8498
I20260807 20:16:45.721721 23131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:16:45 dist-test-slave-2j7r krb5kdc[22976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:16:45.727355 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:45.716409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54080 (local address 127.23.200.254:45551)
0807 20:16:45.716589 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:45.716598 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:45.716627 (+ 29us) server_negotiation.cc:407] Connection header received
0807 20:16:45.716729 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:45.716737 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:45.716795 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:45.716940 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:45.717711 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:45.718949 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:45.719937 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:45.720455 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:45.720873 (+ 418us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:45.723279 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:45.723283 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:45.723284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:45.723296 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:45.725235 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:45.725607 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:45.725610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:45.725611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:45.725751 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:45.725937 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:45.725938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:45.725938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:45.726275 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:45.726912 (+ 637us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:45.726994 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 20:16:45.727144 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260807 20:16:45.735106 23131 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:16:45.735621 23131 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:16:45.737846 23131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a: Generated new TSK 0
I20260807 20:16:45.738034 23131 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.200.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
Entry for principal kudu/127.23.200.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab.
I20260807 20:16:45.772080 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45551
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:44451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:45.882714 23152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:45.882917 23152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:45.882941 23152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:45.884028 23152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:45.884061 23152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:45.884086 23152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:16:45.885546 23152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.193.keytab
--principal=kudu/127.23.200.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:45.885751 23152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:45.885936 23152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:45.890851 23164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:45.890838 23161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:45.890872 23159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:45.891050 23152 server_base.cc:1034] running on GCE node
I20260807 20:16:45.891227 23152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:45.891528 23152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:45.892709 23152 hybrid_clock.cc:648] HybridClock initialized: now 1786133805892686 us; error 31 us; skew 500 ppm
Aug 07 20:16:45 dist-test-slave-2j7r krb5kdc[22976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133805, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:45.895138 23152 init.cc:377] Logged in from keytab as kudu/127.23.200.193@KRBTEST.COM (short username kudu)
I20260807 20:16:45.895604 23152 webserver.cc:460] Webserver started at http://127.23.200.193:33387/ using document root <none> and password file <none>
I20260807 20:16:45.895807 23152 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:45.895854 23152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:45.895982 23152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:45.897034 23152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "6c0ab3c2d80841319a452854b358561a"
format_stamp: "Formatted at 2026-08-07 20:16:45 on dist-test-slave-2j7r"
server_key: "011325aad0457d5bae35612365f96233"
server_key_iv: "a3c24e607efd4b23f5d22608098f49d0"
server_key_version: "encryptionkey@0"
I20260807 20:16:45.897432 23152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "6c0ab3c2d80841319a452854b358561a"
format_stamp: "Formatted at 2026-08-07 20:16:45 on dist-test-slave-2j7r"
server_key: "011325aad0457d5bae35612365f96233"
server_key_iv: "a3c24e607efd4b23f5d22608098f49d0"
server_key_version: "encryptionkey@0"
I20260807 20:16:45.898689 23152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:45.899443 23173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:45.899605 23152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:45.899694 23152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "6c0ab3c2d80841319a452854b358561a"
format_stamp: "Formatted at 2026-08-07 20:16:45 on dist-test-slave-2j7r"
server_key: "011325aad0457d5bae35612365f96233"
server_key_iv: "a3c24e607efd4b23f5d22608098f49d0"
server_key_version: "encryptionkey@0"
I20260807 20:16:45.899760 23152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:45.906378 23152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:45.908212 23152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:45.908344 23152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:45.908592 23152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:45.908880 23152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:45.908916 23152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:45.908944 23152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:45.908970 23152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:45.914731 23152 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:34207
I20260807 20:16:45.914848 23342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:34207 every 8 connection(s)
I20260807 20:16:45.915102 23152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:16:45.918536 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 23152
I20260807 20:16:45.918654 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:16:45.918851 24355 external_mini_cluster.cc:1428] Setting key 2b390f80fa6f5771841f4b094fd34819
Aug 07 20:16:45 dist-test-slave-2j7r krb5kdc[22976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133805, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.193@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:16:45.925316 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:45.915834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:39605 (local address 127.23.200.254:45551)
0807 20:16:45.915898 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:45.915900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:45.916111 (+ 211us) server_negotiation.cc:407] Connection header received
0807 20:16:45.916611 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:45.916612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:45.916633 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:45.916688 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:45.917676 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:45.918383 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:45.919793 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:45.919992 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:45.920041 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:45.922582 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:45.922584 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:45.922584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:45.922593 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:45.923912 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:45.924353 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:45.924355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:45.924355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:45.924411 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:45.924649 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:45.924650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:45.924650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:45.924766 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:45.924824 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:45.925133 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 20:16:45.925225 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 20:16:45.925480 23348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:45.915821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45551 (local address 127.23.200.193:39605)
0807 20:16:45.915970 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:45.916006 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:45.916499 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:45.916755 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:45.916758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:45.917368 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:45.917571 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:45.917575 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:45.918476 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:45.918477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:45.919681 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:45.919685 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:45.920069 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:45.920070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:45.920202 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:45.920694 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:16:45.920705 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:45.922481 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:45.924000 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:45.924009 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:45.924011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:45.924284 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:45.924475 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:45.924476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:45.924477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:45.924590 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:45.924824 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:45.924832 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:45.924980 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 20:16:45.925071 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 20:16:45.925258 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260807 20:16:45.925941 23345 heartbeater.cc:347] Connected to a master server at 127.23.200.254:45551
I20260807 20:16:45.926069 23345 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:45.926285 23345 heartbeater.cc:511] Master 127.23.200.254:45551 requested a full tablet report, sending...
I20260807 20:16:45.926921 23045 ts_manager.cc:194] Registered new tserver with Master: 6c0ab3c2d80841319a452854b358561a (127.23.200.193:34207)
I20260807 20:16:45.927865 23045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.193@KRBTEST.COM'} at 127.23.200.193:39605
WARNING: no policy specified for kudu/127.23.200.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
Entry for principal kudu/127.23.200.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab.
I20260807 20:16:45.949568 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45551
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:44451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:46.034577 23356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:46.034778 23356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:46.034801 23356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:46.035953 23356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:46.035996 23356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:46.036022 23356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:16:46.037489 23356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.194.keytab
--principal=kudu/127.23.200.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:46.037698 23356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:46.037874 23356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:46.042922 23369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:46.042973 23365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:46.043007 23356 server_base.cc:1034] running on GCE node
W20260807 20:16:46.042999 23366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:46.043295 23356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:46.043519 23356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:46.044672 23356 hybrid_clock.cc:648] HybridClock initialized: now 1786133806044657 us; error 28 us; skew 500 ppm
Aug 07 20:16:46 dist-test-slave-2j7r krb5kdc[22976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133806, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:46.047220 23356 init.cc:377] Logged in from keytab as kudu/127.23.200.194@KRBTEST.COM (short username kudu)
I20260807 20:16:46.047704 23356 webserver.cc:460] Webserver started at http://127.23.200.194:37755/ using document root <none> and password file <none>
I20260807 20:16:46.047938 23356 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:46.047986 23356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:46.048112 23356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:46.049185 23356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "5128a892f71d4cc9b2419e41d482b8ed"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "a3089d6ae5a1912b8557b00fca990afb"
server_key_iv: "bac8487284ee68e45d0b1fe6ec3ba40a"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.049518 23356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "5128a892f71d4cc9b2419e41d482b8ed"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "a3089d6ae5a1912b8557b00fca990afb"
server_key_iv: "bac8487284ee68e45d0b1fe6ec3ba40a"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.050740 23356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:46.051406 23376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:46.051576 23356 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:16:46.051664 23356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "5128a892f71d4cc9b2419e41d482b8ed"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "a3089d6ae5a1912b8557b00fca990afb"
server_key_iv: "bac8487284ee68e45d0b1fe6ec3ba40a"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.051723 23356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:46.066016 23356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:46.068213 23356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:46.068361 23356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:46.068598 23356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:46.068912 23356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:46.068946 23356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:46.068977 23356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:46.069000 23356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:46.074890 23356 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:38973
I20260807 20:16:46.074949 23577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:38973 every 8 connection(s)
I20260807 20:16:46.075286 23356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:16:46.075577 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 23356
I20260807 20:16:46.075695 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:16:46.075919 24355 external_mini_cluster.cc:1428] Setting key 8922b740cf8bbb01af7d9a25e0b320d1
Aug 07 20:16:46 dist-test-slave-2j7r krb5kdc[22976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133806, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.194@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.200.195@KRBTEST.COM; defaulting to no policy
I20260807 20:16:46.085005 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.076104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:58805 (local address 127.23.200.254:45551)
0807 20:16:46.076154 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.076156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.076407 (+ 251us) server_negotiation.cc:407] Connection header received
0807 20:16:46.077024 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.077024 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.077036 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.077080 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:46.078437 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.078938 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.079938 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.080137 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.080192 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.082366 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:46.082367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:46.082368 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:46.082377 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:46.083673 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:46.084056 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:46.084057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:46.084058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:46.084114 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:46.084332 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:46.084333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:46.084333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:46.084454 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:46.084512 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.084802 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.084904 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:16:46.085115 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.076119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45551 (local address 127.23.200.194:58805)
0807 20:16:46.076288 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:46.076327 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:46.076926 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:46.077192 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:46.077196 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:46.078051 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:46.078332 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.078339 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.079038 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.079040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.079852 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.079855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.080208 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.080209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.080335 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.080843 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:16:46.080851 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:46.082275 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:46.083750 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:46.083758 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:46.083761 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:46.083988 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:46.084170 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:46.084170 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:46.084171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:46.084275 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:46.084510 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:46.084517 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:46.084656 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 20:16:46.084759 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 20:16:46.084908 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260807 20:16:46.085559 23579 heartbeater.cc:347] Connected to a master server at 127.23.200.254:45551
I20260807 20:16:46.085651 23579 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:46.085825 23579 heartbeater.cc:511] Master 127.23.200.254:45551 requested a full tablet report, sending...
I20260807 20:16:46.086176 23045 ts_manager.cc:194] Registered new tserver with Master: 5128a892f71d4cc9b2419e41d482b8ed (127.23.200.194:38973)
I20260807 20:16:46.086544 23045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.194@KRBTEST.COM'} at 127.23.200.194:58805
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
Entry for principal kudu/127.23.200.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab.
I20260807 20:16:46.102329 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45551
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:44451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:46.184861 23585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:46.185055 23585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:46.185102 23585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:46.186228 23585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:46.186270 23585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:46.186312 23585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:16:46.187741 23585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.195.keytab
--principal=kudu/127.23.200.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:45551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:46.187981 23585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:46.188184 23585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:16:46.193203 23585 server_base.cc:1034] running on GCE node
W20260807 20:16:46.193185 23599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:46.193138 23593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:46.193143 23592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:46.193524 23585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:46.193745 23585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:46.194901 23585 hybrid_clock.cc:648] HybridClock initialized: now 1786133806194879 us; error 37 us; skew 500 ppm
Aug 07 20:16:46 dist-test-slave-2j7r krb5kdc[22976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133806, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:46.197428 23585 init.cc:377] Logged in from keytab as kudu/127.23.200.195@KRBTEST.COM (short username kudu)
I20260807 20:16:46.197872 23585 webserver.cc:460] Webserver started at http://127.23.200.195:38087/ using document root <none> and password file <none>
I20260807 20:16:46.198077 23585 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:46.198117 23585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:46.198207 23585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:46.199222 23585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "20e341970cfe4c35b9af414f2e7f962e"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "3aba940dc190e29929f75b25e3fa6c8d"
server_key_iv: "8ae1c2a5c9c4569e69c429084b3922cb"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.199532 23585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "20e341970cfe4c35b9af414f2e7f962e"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "3aba940dc190e29929f75b25e3fa6c8d"
server_key_iv: "8ae1c2a5c9c4569e69c429084b3922cb"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.200736 23585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 20:16:46.201490 23611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:46.201747 23585 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:16:46.201853 23585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "20e341970cfe4c35b9af414f2e7f962e"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "3aba940dc190e29929f75b25e3fa6c8d"
server_key_iv: "8ae1c2a5c9c4569e69c429084b3922cb"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.201918 23585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:46.208796 23585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:46.210637 23585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:46.210777 23585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:46.210985 23585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:46.211279 23585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:46.211320 23585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:46.211351 23585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:46.211380 23585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:46.217017 23585 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:39807
I20260807 20:16:46.217072 23801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:39807 every 8 connection(s)
I20260807 20:16:46.217465 23585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
Aug 07 20:16:46 dist-test-slave-2j7r krb5kdc[22976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133806, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.195@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:16:46.227025 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.218217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:53855 (local address 127.23.200.254:45551)
0807 20:16:46.218267 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.218270 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.218456 (+ 186us) server_negotiation.cc:407] Connection header received
0807 20:16:46.218999 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.218999 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.219010 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.219068 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:46.220031 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.220545 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.221645 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.221853 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.221920 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.224103 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:46.224105 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:46.224106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:46.224114 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:46.225535 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:46.225942 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:46.225943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:46.225944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:46.226016 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:46.226344 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:46.226345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:46.226345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:46.226453 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:46.226502 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.226842 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.226929 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:16:46.227161 23805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.218163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:45551 (local address 127.23.200.195:53855)
0807 20:16:46.218322 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:46.218365 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:46.218906 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:46.219135 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:46.219138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:46.219740 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:46.219945 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.219951 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.220639 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.220640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.221518 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.221521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.221932 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.221933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.222036 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.222541 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:16:46.222551 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:46.224012 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:46.225616 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:46.225623 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:46.225625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:46.225853 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:46.226105 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:16:46.226106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:16:46.226106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:16:46.226233 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:16:46.226510 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:46.226516 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:46.226648 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 20:16:46.226764 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 20:16:46.226941 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260807 20:16:46.227662 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 23585
I20260807 20:16:46.227687 23803 heartbeater.cc:347] Connected to a master server at 127.23.200.254:45551
I20260807 20:16:46.227756 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:16:46.227775 23803 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:46.227939 24355 external_mini_cluster.cc:1428] Setting key 1090be27ebbac8b303dd710fc9d046a7
I20260807 20:16:46.227964 23803 heartbeater.cc:511] Master 127.23.200.254:45551 requested a full tablet report, sending...
I20260807 20:16:46.228396 23045 ts_manager.cc:194] Registered new tserver with Master: 20e341970cfe4c35b9af414f2e7f962e (127.23.200.195:39807)
I20260807 20:16:46.228796 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:16:46.228870 23045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.200.195@KRBTEST.COM'} at 127.23.200.195:53855
I20260807 20:16:46.235960 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.230122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54084 (local address 127.23.200.254:45551)
0807 20:16:46.230188 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.230190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.230261 (+ 71us) server_negotiation.cc:407] Connection header received
0807 20:16:46.230303 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.230303 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.230312 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.230354 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:46.230815 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.231268 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.231966 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.232141 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.232192 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.233488 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:46.233490 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:46.233490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:46.233497 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:46.234845 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:46.235207 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:46.235208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:46.235208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:46.235259 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:46.235477 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:46.235478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:46.235478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:46.235628 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:46.235706 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.235800 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.235883 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:16:46.237548 23045 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:16:46.239120 23045 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:16:46.245589 23828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.241406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:34207 (local address 127.0.0.1:33456)
0807 20:16:46.241853 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:46.241861 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:46.241911 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:46.242150 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:46.242151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:46.242163 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:46.242219 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.242220 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.243176 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.243177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.244439 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.244443 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.245378 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.245378 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.245432 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.245436 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:16:46.245491 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 20:16:46.245522 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":5,"thread_start_us":80,"threads_started":1}
I20260807 20:16:46.245635 23830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.241595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33456 (local address 127.23.200.193:34207)
0807 20:16:46.241811 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.241817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.241923 (+ 106us) server_negotiation.cc:407] Connection header received
0807 20:16:46.241980 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.241983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.242002 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.242074 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:46.242312 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.243082 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.244525 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.245282 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.245367 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.245427 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.245507 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.245548 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":75,"threads_started":1}
I20260807 20:16:46.245679 23829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.241407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:38973 (local address 127.0.0.1:53616)
0807 20:16:46.241978 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:46.241986 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:46.242020 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:46.242361 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:46.242362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:46.242364 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:46.242397 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.242399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.243577 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.243578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.244450 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.244453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.245463 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.245464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.245525 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.245529 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:16:46.245575 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 20:16:46.245617 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"mutex_wait_us":22,"thread_start_us":103,"threads_started":1}
I20260807 20:16:46.245904 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.241664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53616 (local address 127.23.200.194:38973)
0807 20:16:46.242139 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.242143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.242152 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:16:46.242186 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.242188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.242202 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.242271 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:46.242481 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.243492 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.244545 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.245315 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.245373 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.245427 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.245771 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.245821 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":85,"threads_started":1}
I20260807 20:16:46.246629 23457 tablet_service.cc:1467] Processing CreateTablet for tablet c12a26ca15d8409dabc93ff5a83ce337 (DEFAULT_TABLE table=test-table [id=2c7e4c6eb17d4554a0038f70738da93f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:46.246999 23457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c12a26ca15d8409dabc93ff5a83ce337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:46.247925 23247 tablet_service.cc:1467] Processing CreateTablet for tablet c12a26ca15d8409dabc93ff5a83ce337 (DEFAULT_TABLE table=test-table [id=2c7e4c6eb17d4554a0038f70738da93f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:46.248287 23247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c12a26ca15d8409dabc93ff5a83ce337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:46.246125 23834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.241657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53256 (local address 127.23.200.195:39807)
0807 20:16:46.241851 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.241856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.241866 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:16:46.241897 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.241898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.241919 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.241996 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:46.242243 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.243352 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.244586 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.245119 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.245184 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.245243 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.245569 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.245617 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":91,"threads_started":1}
I20260807 20:16:46.245501 23826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.241406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:39807 (local address 127.0.0.1:53256)
0807 20:16:46.241564 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:46.241605 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:46.241675 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:46.242017 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:46.242021 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:46.242023 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:46.242091 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.242093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.243453 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.243454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.244528 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.244530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.245194 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.245195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.245285 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.245299 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 20:16:46.245364 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 20:16:46.245419 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":24,"thread_start_us":52,"threads_started":1}
I20260807 20:16:46.250120 23837 tablet_bootstrap.cc:492] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed: Bootstrap starting.
I20260807 20:16:46.250560 23692 tablet_service.cc:1467] Processing CreateTablet for tablet c12a26ca15d8409dabc93ff5a83ce337 (DEFAULT_TABLE table=test-table [id=2c7e4c6eb17d4554a0038f70738da93f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:46.250697 23837 tablet_bootstrap.cc:654] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:46.250867 23692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c12a26ca15d8409dabc93ff5a83ce337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:46.251107 23837 log.cc:824] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:46.251783 23837 tablet_bootstrap.cc:492] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed: No bootstrap required, opened a new log
I20260807 20:16:46.251830 23837 ts_tablet_manager.cc:1397] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:16:46.253317 23839 tablet_bootstrap.cc:492] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a: Bootstrap starting.
I20260807 20:16:46.253316 23837 raft_consensus.cc:348] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0ab3c2d80841319a452854b358561a" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34207 } } peers { permanent_uuid: "20e341970cfe4c35b9af414f2e7f962e" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 39807 } } peers { permanent_uuid: "5128a892f71d4cc9b2419e41d482b8ed" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38973 } }
I20260807 20:16:46.253491 23837 raft_consensus.cc:374] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:46.253525 23837 raft_consensus.cc:729] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5128a892f71d4cc9b2419e41d482b8ed, State: Initialized, Role: FOLLOWER
I20260807 20:16:46.253561 23840 tablet_bootstrap.cc:492] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e: Bootstrap starting.
I20260807 20:16:46.253758 23837 consensus_queue.cc:260] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0ab3c2d80841319a452854b358561a" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34207 } } peers { permanent_uuid: "20e341970cfe4c35b9af414f2e7f962e" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 39807 } } peers { permanent_uuid: "5128a892f71d4cc9b2419e41d482b8ed" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38973 } }
I20260807 20:16:46.253983 23839 tablet_bootstrap.cc:654] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:46.254043 23837 ts_tablet_manager.cc:1428] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:16:46.254180 23579 heartbeater.cc:503] Master 127.23.200.254:45551 was elected leader, sending a full tablet report...
I20260807 20:16:46.254199 23840 tablet_bootstrap.cc:654] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:46.254563 23839 log.cc:824] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:46.254745 23840 log.cc:824] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:46.255304 23839 tablet_bootstrap.cc:492] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a: No bootstrap required, opened a new log
I20260807 20:16:46.255370 23839 ts_tablet_manager.cc:1397] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:16:46.255524 23840 tablet_bootstrap.cc:492] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e: No bootstrap required, opened a new log
I20260807 20:16:46.255589 23840 ts_tablet_manager.cc:1397] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:16:46.256723 23839 raft_consensus.cc:348] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0ab3c2d80841319a452854b358561a" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34207 } } peers { permanent_uuid: "20e341970cfe4c35b9af414f2e7f962e" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 39807 } } peers { permanent_uuid: "5128a892f71d4cc9b2419e41d482b8ed" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38973 } }
I20260807 20:16:46.256860 23839 raft_consensus.cc:374] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:46.256822 23840 raft_consensus.cc:348] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0ab3c2d80841319a452854b358561a" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34207 } } peers { permanent_uuid: "20e341970cfe4c35b9af414f2e7f962e" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 39807 } } peers { permanent_uuid: "5128a892f71d4cc9b2419e41d482b8ed" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38973 } }
I20260807 20:16:46.256910 23839 raft_consensus.cc:729] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c0ab3c2d80841319a452854b358561a, State: Initialized, Role: FOLLOWER
I20260807 20:16:46.256937 23840 raft_consensus.cc:374] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:46.256956 23840 raft_consensus.cc:729] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20e341970cfe4c35b9af414f2e7f962e, State: Initialized, Role: FOLLOWER
I20260807 20:16:46.257032 23839 consensus_queue.cc:260] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0ab3c2d80841319a452854b358561a" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34207 } } peers { permanent_uuid: "20e341970cfe4c35b9af414f2e7f962e" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 39807 } } peers { permanent_uuid: "5128a892f71d4cc9b2419e41d482b8ed" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38973 } }
I20260807 20:16:46.257225 23840 consensus_queue.cc:260] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0ab3c2d80841319a452854b358561a" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34207 } } peers { permanent_uuid: "20e341970cfe4c35b9af414f2e7f962e" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 39807 } } peers { permanent_uuid: "5128a892f71d4cc9b2419e41d482b8ed" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38973 } }
I20260807 20:16:46.257331 23839 ts_tablet_manager.cc:1428] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:16:46.257450 23345 heartbeater.cc:503] Master 127.23.200.254:45551 was elected leader, sending a full tablet report...
I20260807 20:16:46.257524 23840 ts_tablet_manager.cc:1428] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:16:46.257638 23803 heartbeater.cc:503] Master 127.23.200.254:45551 was elected leader, sending a full tablet report...
W20260807 20:16:46.325925 23580 tablet.cc:2369] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:16:46.348246 23841 raft_consensus.cc:482] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:16:46.348387 23841 raft_consensus.cc:504] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0ab3c2d80841319a452854b358561a" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34207 } } peers { permanent_uuid: "20e341970cfe4c35b9af414f2e7f962e" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 39807 } } peers { permanent_uuid: "5128a892f71d4cc9b2419e41d482b8ed" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38973 } }
I20260807 20:16:46.348728 23841 leader_election.cc:288] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c0ab3c2d80841319a452854b358561a (127.23.200.193:34207), 20e341970cfe4c35b9af414f2e7f962e (127.23.200.195:39807)
I20260807 20:16:46.351423 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.348916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:39807 (local address 127.23.200.194:38177)
0807 20:16:46.348979 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:46.348991 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:46.349041 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:46.349222 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:46.349223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:46.349224 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:46.349291 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.349293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.349932 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.349934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.350652 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.350654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.351194 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.351195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.351259 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.351262 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:16:46.351308 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 20:16:46.351346 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 20:16:46.351425 23834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.349054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:38177 (local address 127.23.200.195:39807)
0807 20:16:46.349118 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.349120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.349128 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:16:46.349154 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.349155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.349162 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.349206 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:46.349399 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.349852 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.350764 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.351121 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.351182 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.351191 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.351320 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.351363 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:16:46.351658 23830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.349000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:36417 (local address 127.23.200.193:34207)
0807 20:16:46.349049 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.349051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.349306 (+ 255us) server_negotiation.cc:407] Connection header received
0807 20:16:46.349343 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.349344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.349354 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.349441 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:46.349582 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.350045 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.350928 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.351302 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.351364 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.351372 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.351540 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.351589 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:16:46.351758 23726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c12a26ca15d8409dabc93ff5a83ce337" candidate_uuid: "5128a892f71d4cc9b2419e41d482b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20e341970cfe4c35b9af414f2e7f962e" is_pre_election: true
I20260807 20:16:46.351897 23848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.348923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:34207 (local address 127.23.200.194:36417)
0807 20:16:46.349226 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:46.349235 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:46.349297 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:46.349448 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:46.349448 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:46.349450 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:46.349519 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.349520 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.350127 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.350128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.350838 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.350840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.351385 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.351386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.351452 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.351455 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:16:46.351776 (+ 321us) client_negotiation.cc:240] Negotiation successful
0807 20:16:46.351817 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":81,"thread_start_us":153,"threads_started":1}
I20260807 20:16:46.351922 23726 raft_consensus.cc:2454] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5128a892f71d4cc9b2419e41d482b8ed in term 0.
I20260807 20:16:46.352144 23380 leader_election.cc:302] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20e341970cfe4c35b9af414f2e7f962e, 5128a892f71d4cc9b2419e41d482b8ed; no voters:
I20260807 20:16:46.352206 23273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c12a26ca15d8409dabc93ff5a83ce337" candidate_uuid: "5128a892f71d4cc9b2419e41d482b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c0ab3c2d80841319a452854b358561a" is_pre_election: true
I20260807 20:16:46.352267 23841 raft_consensus.cc:2781] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:16:46.352312 23841 raft_consensus.cc:482] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:16:46.352341 23841 raft_consensus.cc:3037] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:46.352362 23273 raft_consensus.cc:2454] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5128a892f71d4cc9b2419e41d482b8ed in term 0.
I20260807 20:16:46.353071 23841 raft_consensus.cc:504] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0ab3c2d80841319a452854b358561a" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34207 } } peers { permanent_uuid: "20e341970cfe4c35b9af414f2e7f962e" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 39807 } } peers { permanent_uuid: "5128a892f71d4cc9b2419e41d482b8ed" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38973 } }
I20260807 20:16:46.353223 23841 leader_election.cc:288] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [CANDIDATE]: Term 1 election: Requested vote from peers 6c0ab3c2d80841319a452854b358561a (127.23.200.193:34207), 20e341970cfe4c35b9af414f2e7f962e (127.23.200.195:39807)
I20260807 20:16:46.353370 23726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c12a26ca15d8409dabc93ff5a83ce337" candidate_uuid: "5128a892f71d4cc9b2419e41d482b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20e341970cfe4c35b9af414f2e7f962e"
I20260807 20:16:46.353420 23273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c12a26ca15d8409dabc93ff5a83ce337" candidate_uuid: "5128a892f71d4cc9b2419e41d482b8ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c0ab3c2d80841319a452854b358561a"
I20260807 20:16:46.353451 23726 raft_consensus.cc:3037] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:46.353493 23273 raft_consensus.cc:3037] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:46.354179 23273 raft_consensus.cc:2454] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5128a892f71d4cc9b2419e41d482b8ed in term 1.
I20260807 20:16:46.354188 23726 raft_consensus.cc:2454] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5128a892f71d4cc9b2419e41d482b8ed in term 1.
I20260807 20:16:46.354354 23381 leader_election.cc:302] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5128a892f71d4cc9b2419e41d482b8ed, 6c0ab3c2d80841319a452854b358561a; no voters:
I20260807 20:16:46.354470 23841 raft_consensus.cc:2781] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:46.354606 23841 raft_consensus.cc:686] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [term 1 LEADER]: Becoming Leader. State: Replica: 5128a892f71d4cc9b2419e41d482b8ed, State: Running, Role: LEADER
I20260807 20:16:46.354686 23841 consensus_queue.cc:237] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0ab3c2d80841319a452854b358561a" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34207 } } peers { permanent_uuid: "20e341970cfe4c35b9af414f2e7f962e" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 39807 } } peers { permanent_uuid: "5128a892f71d4cc9b2419e41d482b8ed" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38973 } }
I20260807 20:16:46.355530 23045 catalog_manager.cc:5495] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 5128a892f71d4cc9b2419e41d482b8ed (127.23.200.194). New cstate: current_term: 1 leader_uuid: "5128a892f71d4cc9b2419e41d482b8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c0ab3c2d80841319a452854b358561a" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20e341970cfe4c35b9af414f2e7f962e" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 39807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5128a892f71d4cc9b2419e41d482b8ed" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 38973 } health_report { overall_health: HEALTHY } } }
I20260807 20:16:46.360183 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.358045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53630 (local address 127.23.200.194:38973)
0807 20:16:46.358085 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.358087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.358096 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:16:46.358158 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.358159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.358173 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.358219 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:16:46.358301 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.358741 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.359462 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.359656 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.359716 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.359914 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:16:46.360009 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:16:46.360075 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.360117 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.360135 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:16:46.361794 23726 raft_consensus.cc:1261] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e [term 1 FOLLOWER]: Refusing update from remote peer 5128a892f71d4cc9b2419e41d482b8ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:16:46.361800 23273 raft_consensus.cc:1261] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a [term 1 FOLLOWER]: Refusing update from remote peer 5128a892f71d4cc9b2419e41d482b8ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:16:46.362097 23849 consensus_queue.cc:1035] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c0ab3c2d80841319a452854b358561a" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:16:46.362214 23841 consensus_queue.cc:1035] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "20e341970cfe4c35b9af414f2e7f962e" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 39807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:16:46.364811 23853 mvcc.cc:205] Tried to move back new op lower bound from 7316004070855643136 to 7316004070829293568. Current Snapshot: MvccSnapshot[applied={T|T < 7316004070855643136}]
I20260807 20:16:46.365267 23852 mvcc.cc:205] Tried to move back new op lower bound from 7316004070855643136 to 7316004070829293568. Current Snapshot: MvccSnapshot[applied={T|T < 7316004070855643136 or (T in {7316004070855643136})}]
I20260807 20:16:46.366719 23045 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:16:46.366801 23045 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:16:46.367506 23045 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ca65b5eae4b461ebf3edf1c40fd622a: Sending DeleteTablet for 3 replicas of tablet c12a26ca15d8409dabc93ff5a83ce337
I20260807 20:16:46.367841 23247 tablet_service.cc:1514] Processing DeleteTablet for tablet c12a26ca15d8409dabc93ff5a83ce337 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:46 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:33456
I20260807 20:16:46.367928 23692 tablet_service.cc:1514] Processing DeleteTablet for tablet c12a26ca15d8409dabc93ff5a83ce337 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:46 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:53256
I20260807 20:16:46.368266 23874 tablet_replica.cc:330] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a: stopping tablet replica
I20260807 20:16:46.368252 23457 tablet_service.cc:1514] Processing DeleteTablet for tablet c12a26ca15d8409dabc93ff5a83ce337 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:46 UTC) from {username='kudu', principal='kudu/127.23.200.254@KRBTEST.COM'} at 127.0.0.1:53616
I20260807 20:16:46.368377 23874 raft_consensus.cc:2229] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:16:46.368474 23874 raft_consensus.cc:2258] T c12a26ca15d8409dabc93ff5a83ce337 P 6c0ab3c2d80841319a452854b358561a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:46.368528 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 23152
I20260807 20:16:46.369168 23876 tablet_replica.cc:330] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed: stopping tablet replica
I20260807 20:16:46.369274 23876 raft_consensus.cc:2229] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:16:46.369508 23876 raft_consensus.cc:2258] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:46.372143 23876 ts_tablet_manager.cc:1905] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:16:46.374017 23876 ts_tablet_manager.cc:1918] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:16:46.374110 23876 log.cc:1196] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/c12a26ca15d8409dabc93ff5a83ce337
I20260807 20:16:46.374444 23876 ts_tablet_manager.cc:1939] T c12a26ca15d8409dabc93ff5a83ce337 P 5128a892f71d4cc9b2419e41d482b8ed: Deleting consensus metadata
I20260807 20:16:46.374948 23029 catalog_manager.cc:4841] TS 5128a892f71d4cc9b2419e41d482b8ed (127.23.200.194:38973): tablet c12a26ca15d8409dabc93ff5a83ce337 (table test-table [id=2c7e4c6eb17d4554a0038f70738da93f]) successfully deleted
W20260807 20:16:46.383309 23026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:34207 (error 108)
I20260807 20:16:46.368139 23875 tablet_replica.cc:330] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e: stopping tablet replica
I20260807 20:16:46.383581 23875 raft_consensus.cc:2229] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:16:46.383728 23875 raft_consensus.cc:2258] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:46.383723 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 23356
I20260807 20:16:46.386359 23826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.383804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:34207 (local address 127.0.0.1:33466)
0807 20:16:46.386264 (+ 2460us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:46.386317 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:34207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2433}
W20260807 20:16:46.386494 23026 catalog_manager.cc:4584] TS 6c0ab3c2d80841319a452854b358561a (127.23.200.193:34207): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c12a26ca15d8409dabc93ff5a83ce337: Network error: Client connection negotiation failed: client connection to 127.23.200.193:34207: connect: Connection refused (error 111)
I20260807 20:16:46.391127 23875 ts_tablet_manager.cc:1905] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:16:46.392933 23875 ts_tablet_manager.cc:1918] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:16:46.393016 23875 log.cc:1196] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/c12a26ca15d8409dabc93ff5a83ce337
I20260807 20:16:46.393262 23875 ts_tablet_manager.cc:1939] T c12a26ca15d8409dabc93ff5a83ce337 P 20e341970cfe4c35b9af414f2e7f962e: Deleting consensus metadata
I20260807 20:16:46.393796 23025 catalog_manager.cc:4841] TS 20e341970cfe4c35b9af414f2e7f962e (127.23.200.195:39807): tablet c12a26ca15d8409dabc93ff5a83ce337 (table test-table [id=2c7e4c6eb17d4554a0038f70738da93f]) successfully deleted
I20260807 20:16:46.394913 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 23585
I20260807 20:16:46.403668 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 22998
2026-08-07T20:16:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2118 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T20:16:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:16:46Z Disabled control of system clock
I20260807 20:16:46.438898 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:33203
--webserver_interface=127.23.200.254
--webserver_port=0
--builtin_ntp_servers=127.23.200.212:34795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:33203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:16:46.521456 23886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:46.521682 23886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:46.521714 23886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:46.522847 23886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:16:46.522889 23886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:46.522905 23886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:46.522921 23886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:16:46.522935 23886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:16:46.524255 23886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:33203
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:33203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:46.524502 23886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:46.524777 23886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:46.529599 23894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:46.529583 23898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:46.529605 23886 server_base.cc:1034] running on GCE node
W20260807 20:16:46.529588 23896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:46.530032 23886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:46.530283 23886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:46.531455 23886 hybrid_clock.cc:648] HybridClock initialized: now 1786133806531436 us; error 34 us; skew 500 ppm
I20260807 20:16:46.532722 23886 webserver.cc:460] Webserver started at http://127.23.200.254:43081/ using document root <none> and password file <none>
I20260807 20:16:46.532955 23886 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:46.533006 23886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:46.533174 23886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:46.534374 23886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "4bbeea346c7841aa8d816a1a5e09b39f"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "8066c38003428b8db7e7ad77a1bba92e"
server_key_iv: "28bede27dfed86bd25e06e183664bbc7"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.534821 23886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "4bbeea346c7841aa8d816a1a5e09b39f"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "8066c38003428b8db7e7ad77a1bba92e"
server_key_iv: "28bede27dfed86bd25e06e183664bbc7"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.536203 23886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:16:46.536917 23914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:46.537071 23886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:16:46.537199 23886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "4bbeea346c7841aa8d816a1a5e09b39f"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "8066c38003428b8db7e7ad77a1bba92e"
server_key_iv: "28bede27dfed86bd25e06e183664bbc7"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.537276 23886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:46.544670 23886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:46.544958 23886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:46.545120 23886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:46.549186 23886 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:33203
I20260807 20:16:46.549243 24009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:33203 every 8 connection(s)
I20260807 20:16:46.549568 23886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:16:46.550547 24011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:46.552845 24011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f: Bootstrap starting.
I20260807 20:16:46.553517 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 23886
I20260807 20:16:46.553565 24011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:46.553638 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:16:46.553853 24355 external_mini_cluster.cc:1428] Setting key aa4ce9aa2968a1a79dcd875d8b918304
I20260807 20:16:46.553898 24011 log.cc:824] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:46.554633 24011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f: No bootstrap required, opened a new log
I20260807 20:16:46.556898 24011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbeea346c7841aa8d816a1a5e09b39f" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33203 } }
I20260807 20:16:46.557075 24011 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:46.557137 24011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bbeea346c7841aa8d816a1a5e09b39f, State: Initialized, Role: FOLLOWER
I20260807 20:16:46.557264 24011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbeea346c7841aa8d816a1a5e09b39f" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33203 } }
I20260807 20:16:46.557335 24011 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:16:46.557363 24011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:16:46.557404 24011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:46.557984 24011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbeea346c7841aa8d816a1a5e09b39f" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33203 } }
I20260807 20:16:46.558085 24011 leader_election.cc:302] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bbeea346c7841aa8d816a1a5e09b39f; no voters:
I20260807 20:16:46.558254 24011 leader_election.cc:288] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:16:46.558358 24018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:46.558509 24018 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [term 1 LEADER]: Becoming Leader. State: Replica: 4bbeea346c7841aa8d816a1a5e09b39f, State: Running, Role: LEADER
I20260807 20:16:46.558539 24011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:16:46.558599 24018 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbeea346c7841aa8d816a1a5e09b39f" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33203 } }
I20260807 20:16:46.559002 24020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bbeea346c7841aa8d816a1a5e09b39f. Latest consensus state: current_term: 1 leader_uuid: "4bbeea346c7841aa8d816a1a5e09b39f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbeea346c7841aa8d816a1a5e09b39f" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33203 } } }
I20260807 20:16:46.559087 24020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:46.559288 24023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bbeea346c7841aa8d816a1a5e09b39f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbeea346c7841aa8d816a1a5e09b39f" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33203 } } }
I20260807 20:16:46.559433 24023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:46.559623 24017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.554393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36088 (local address 127.23.200.254:33203)
0807 20:16:46.554579 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.554586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.554612 (+ 26us) server_negotiation.cc:407] Connection header received
0807 20:16:46.555074 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.555082 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.555145 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.555280 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:46.555575 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.556522 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.557589 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.557999 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.558125 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.558808 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:46.558812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:46.558812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:46.559016 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:46.559064 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:46.559080 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:46.559150 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.559355 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.559438 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":75,"threads_started":1}
W20260807 20:16:46.560305 24047 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:16:46.560410 24047 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:16:46.560461 24034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:16:46.560612 24034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:16:46.562085 24034 catalog_manager.cc:1347] Generated new cluster ID: 86bf76e0d11d42518a21b734320861d9
I20260807 20:16:46.562143 24034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:16:46.575605 24034 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:16:46.576153 24034 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:16:46.580428 24034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f: Generated new TSK 0
I20260807 20:16:46.580606 24034 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:16:46.583680 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.193:0
--local_ip_for_outbound_sockets=127.23.200.193
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:33203
--builtin_ntp_servers=127.23.200.212:34795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:16:46.665663 24055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:46.665867 24055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:46.665889 24055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:46.667039 24055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:46.667079 24055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:46.667104 24055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.193
I20260807 20:16:46.668485 24055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.200.193
--webserver_port=0
--tserver_master_addrs=127.23.200.254:33203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.193
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:46.668716 24055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:46.669003 24055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:46.674942 24061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:46.674974 24062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:46.674978 24064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:46.675283 24055 server_base.cc:1034] running on GCE node
I20260807 20:16:46.675527 24055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:46.675807 24055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:46.676985 24055 hybrid_clock.cc:648] HybridClock initialized: now 1786133806676963 us; error 35 us; skew 500 ppm
I20260807 20:16:46.678581 24055 webserver.cc:460] Webserver started at http://127.23.200.193:42373/ using document root <none> and password file <none>
I20260807 20:16:46.679024 24055 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:46.679097 24055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:46.679234 24055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:46.680658 24055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/data/instance:
uuid: "065e9036915f4100a9c1e200b07289c2"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "658938a5adbfaab7e26cd119b664e586"
server_key_iv: "6d0badfe27c728be3073851301fd9d85"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.681159 24055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance:
uuid: "065e9036915f4100a9c1e200b07289c2"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "658938a5adbfaab7e26cd119b664e586"
server_key_iv: "6d0badfe27c728be3073851301fd9d85"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.682873 24055 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260807 20:16:46.683764 24077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:46.683936 24055 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:46.684027 24055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/wal
uuid: "065e9036915f4100a9c1e200b07289c2"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "658938a5adbfaab7e26cd119b664e586"
server_key_iv: "6d0badfe27c728be3073851301fd9d85"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.684098 24055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:46.698165 24055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:46.698453 24055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:46.698561 24055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:46.698776 24055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:46.699069 24055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:46.699098 24055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:46.699124 24055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:46.699141 24055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:46.704614 24055 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.193:45007
I20260807 20:16:46.704700 24267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.193:45007 every 8 connection(s)
I20260807 20:16:46.705008 24055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/data/info.pb
I20260807 20:16:46.709518 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 24055
I20260807 20:16:46.709625 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-0/wal/instance
I20260807 20:16:46.709860 24355 external_mini_cluster.cc:1428] Setting key 4fa3128f8795809dc846fb339c4ecfac
I20260807 20:16:46.710367 24017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.705776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.193:47025 (local address 127.23.200.254:33203)
0807 20:16:46.705841 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.705843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.706054 (+ 211us) server_negotiation.cc:407] Connection header received
0807 20:16:46.706640 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.706642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.706655 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.706707 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:46.707401 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.707912 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.708882 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.709107 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.709168 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.710074 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:46.710076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:46.710077 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:46.710082 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:46.710101 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:46.710105 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:46.710155 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.710247 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.710282 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:16:46.710610 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.705757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:33203 (local address 127.23.200.193:47025)
0807 20:16:46.705925 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:46.705970 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:46.706536 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:46.706779 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:46.706782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:46.706790 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:46.707315 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.707320 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.707997 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.707998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.708798 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.708801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.709188 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.709189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.709328 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.709902 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:16:46.709902 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:46.709946 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:16:46.709977 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:46.710170 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:46.710177 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:46.710180 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:16:46.710244 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 20:16:46.710388 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260807 20:16:46.710989 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.194:0
--local_ip_for_outbound_sockets=127.23.200.194
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:33203
--builtin_ntp_servers=127.23.200.212:34795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 20:16:46.711174 24268 heartbeater.cc:347] Connected to a master server at 127.23.200.254:33203
I20260807 20:16:46.711272 24268 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:46.711463 24268 heartbeater.cc:511] Master 127.23.200.254:33203 requested a full tablet report, sending...
I20260807 20:16:46.712009 23950 ts_manager.cc:194] Registered new tserver with Master: 065e9036915f4100a9c1e200b07289c2 (127.23.200.193:45007)
I20260807 20:16:46.712841 23950 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.200.193:47025
W20260807 20:16:46.796520 24274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:46.796725 24274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:46.796754 24274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:46.797922 24274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:46.797963 24274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:46.797994 24274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.194
I20260807 20:16:46.799319 24274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.200.194
--webserver_port=0
--tserver_master_addrs=127.23.200.254:33203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.194
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:46.799556 24274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:46.799840 24274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:46.804827 24280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:46.804873 24279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:46.804855 24282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:46.805032 24274 server_base.cc:1034] running on GCE node
I20260807 20:16:46.805239 24274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:46.805462 24274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:46.806619 24274 hybrid_clock.cc:648] HybridClock initialized: now 1786133806806598 us; error 34 us; skew 500 ppm
I20260807 20:16:46.807998 24274 webserver.cc:460] Webserver started at http://127.23.200.194:35149/ using document root <none> and password file <none>
I20260807 20:16:46.808224 24274 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:46.808274 24274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:46.808396 24274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:46.809546 24274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/data/instance:
uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "bf1cddba7e5639ec59a7b651b43343ee"
server_key_iv: "0363fcc68777b8891283fd0d5560fd5e"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.809885 24274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance:
uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "bf1cddba7e5639ec59a7b651b43343ee"
server_key_iv: "0363fcc68777b8891283fd0d5560fd5e"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.811116 24274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 20:16:46.811893 24292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:46.812062 24274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:16:46.812144 24274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/wal
uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "bf1cddba7e5639ec59a7b651b43343ee"
server_key_iv: "0363fcc68777b8891283fd0d5560fd5e"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.812213 24274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:46.828532 24274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:46.828838 24274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:46.828970 24274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:46.829226 24274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:46.829564 24274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:46.829599 24274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:46.829629 24274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:46.829646 24274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:46.835225 24274 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.194:33219
I20260807 20:16:46.835268 24486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.194:33219 every 8 connection(s)
I20260807 20:16:46.835599 24274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/data/info.pb
I20260807 20:16:46.836997 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 24274
I20260807 20:16:46.837122 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/wal/instance
I20260807 20:16:46.837363 24355 external_mini_cluster.cc:1428] Setting key 9536f790547c13c6738d9c7b9e1969c4
I20260807 20:16:46.838431 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.200.195:0
--local_ip_for_outbound_sockets=127.23.200.195
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:33203
--builtin_ntp_servers=127.23.200.212:34795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 20:16:46.841676 24017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.836325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:57095 (local address 127.23.200.254:33203)
0807 20:16:46.836389 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.836392 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.836688 (+ 296us) server_negotiation.cc:407] Connection header received
0807 20:16:46.837430 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.837431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.837445 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.837514 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:46.838421 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.839019 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.840315 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.840520 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.840576 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.841423 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:46.841424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:46.841425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:46.841434 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:46.841450 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:46.841454 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:46.841501 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.841590 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.841615 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 20:16:46.841903 24495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.836400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:33203 (local address 127.23.200.194:57095)
0807 20:16:46.836567 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:46.836602 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:46.837238 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:46.837650 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:46.837654 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:46.837663 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:46.838307 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.838314 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.839115 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.839117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.840236 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.840239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.840587 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.840588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.840724 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.841245 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:16:46.841245 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:46.841294 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:16:46.841325 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:46.841512 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:46.841517 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:46.841520 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:16:46.841586 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 20:16:46.841688 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260807 20:16:46.842650 24489 heartbeater.cc:347] Connected to a master server at 127.23.200.254:33203
I20260807 20:16:46.842738 24489 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:46.842914 24489 heartbeater.cc:511] Master 127.23.200.254:33203 requested a full tablet report, sending...
I20260807 20:16:46.843324 23950 ts_manager.cc:194] Registered new tserver with Master: 2d71d7abc67f4fd68a7fc59b1fa7fc6a (127.23.200.194:33219)
I20260807 20:16:46.843812 23950 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.200.194:57095
W20260807 20:16:46.923703 24498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:46.923916 24498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:46.923938 24498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:46.925104 24498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:46.925146 24498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:46.925170 24498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.200.195
I20260807 20:16:46.926565 24498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.200.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.200.195
--webserver_port=0
--tserver_master_addrs=127.23.200.254:33203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.200.195
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:46.926787 24498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:46.927045 24498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:46.932086 24507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:46.932090 24506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:46.932093 24510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:46.932281 24498 server_base.cc:1034] running on GCE node
I20260807 20:16:46.932516 24498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:46.932749 24498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:46.933898 24498 hybrid_clock.cc:648] HybridClock initialized: now 1786133806933880 us; error 29 us; skew 500 ppm
I20260807 20:16:46.935202 24498 webserver.cc:460] Webserver started at http://127.23.200.195:40015/ using document root <none> and password file <none>
I20260807 20:16:46.935468 24498 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:46.935520 24498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:46.935650 24498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:46.936723 24498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/data/instance:
uuid: "0e99ba920ed449898be2796236c9a2c5"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "911bd1c5e17bbcb44560db47503c0fa1"
server_key_iv: "b48e882f5cae3bd052477f95a20b6f4e"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.937057 24498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance:
uuid: "0e99ba920ed449898be2796236c9a2c5"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "911bd1c5e17bbcb44560db47503c0fa1"
server_key_iv: "b48e882f5cae3bd052477f95a20b6f4e"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.938441 24498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:46.939214 24519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:46.939440 24498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:16:46.939554 24498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/wal
uuid: "0e99ba920ed449898be2796236c9a2c5"
format_stamp: "Formatted at 2026-08-07 20:16:46 on dist-test-slave-2j7r"
server_key: "911bd1c5e17bbcb44560db47503c0fa1"
server_key_iv: "b48e882f5cae3bd052477f95a20b6f4e"
server_key_version: "encryptionkey@0"
I20260807 20:16:46.939630 24498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:46.950546 24498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:46.950829 24498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:46.950932 24498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:46.951121 24498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:16:46.951418 24498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:16:46.951455 24498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:46.951478 24498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:16:46.951488 24498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:46.956990 24498 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.195:40341
I20260807 20:16:46.957056 24702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.195:40341 every 8 connection(s)
I20260807 20:16:46.957386 24498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/data/info.pb
I20260807 20:16:46.962692 24017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.958187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.195:57591 (local address 127.23.200.254:33203)
0807 20:16:46.958240 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.958297 (+ 57us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.958504 (+ 207us) server_negotiation.cc:407] Connection header received
0807 20:16:46.959114 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.959115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.959127 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.959177 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:46.959876 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.960376 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.961390 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.961589 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.961653 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.962436 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:46.962437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:46.962438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:46.962443 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:46.962458 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:46.962463 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:46.962508 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.962582 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.962610 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:16:46.962925 24707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.958143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:33203 (local address 127.23.200.195:57591)
0807 20:16:46.958383 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:46.958420 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:46.959014 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:46.959249 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:46.959253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:46.959263 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:16:46.959770 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.959774 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.960463 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.960464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.961313 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.961315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.961665 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.961666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.961808 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.962295 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:16:46.962295 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:16:46.962325 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:16:46.962349 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:16:46.962513 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:16:46.962519 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:16:46.962522 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:16:46.962582 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 20:16:46.962687 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":95,"threads_started":1}
I20260807 20:16:46.963611 24703 heartbeater.cc:347] Connected to a master server at 127.23.200.254:33203
I20260807 20:16:46.963723 24703 heartbeater.cc:464] Registering TS with master...
I20260807 20:16:46.963927 24703 heartbeater.cc:511] Master 127.23.200.254:33203 requested a full tablet report, sending...
I20260807 20:16:46.964355 23950 ts_manager.cc:194] Registered new tserver with Master: 0e99ba920ed449898be2796236c9a2c5 (127.23.200.195:40341)
I20260807 20:16:46.964720 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 24498
I20260807 20:16:46.964803 23950 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.200.195:57591
I20260807 20:16:46.964831 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/wal/instance
I20260807 20:16:46.965016 24355 external_mini_cluster.cc:1428] Setting key bb31fbefcb51969e6f4af16d7a16258b
I20260807 20:16:46.965870 24355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:16:46.969880 24017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.967197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34806 (local address 127.23.200.254:33203)
0807 20:16:46.967237 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.967239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.967345 (+ 106us) server_negotiation.cc:407] Connection header received
0807 20:16:46.967394 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.967395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.967404 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.967448 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:46.967557 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.968008 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.968803 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.969046 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.969149 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.969717 (+ 568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:46.969718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:46.969719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:46.969723 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:46.969735 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:46.969739 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:46.969775 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.969805 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.969823 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:16:46.971241 23950 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:16:46.972759 23950 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:16:46.979436 24728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.975052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35024 (local address 127.23.200.193:45007)
0807 20:16:46.975232 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.975237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.975247 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:16:46.975286 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.975296 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.975309 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.975376 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:46.975571 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.976722 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.978252 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.978994 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.979051 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.979119 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.979306 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.979349 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":77,"threads_started":1}
I20260807 20:16:46.979435 24723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.974861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:45007 (local address 127.0.0.1:35024)
0807 20:16:46.975048 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:46.975089 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:46.975155 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:46.975404 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:46.975407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:46.975409 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:46.975485 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.975486 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.976813 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.976814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.978162 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.978166 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.979087 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.979088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.979176 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.979192 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 20:16:46.979276 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 20:16:46.979336 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":7,"thread_start_us":72,"threads_started":1}
I20260807 20:16:46.980211 24725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.974861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:40341 (local address 127.0.0.1:60014)
0807 20:16:46.975360 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:46.975367 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:46.975415 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:46.975689 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:46.975690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:46.975692 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:46.975735 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.975736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.976938 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.976939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.978163 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.978166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.979958 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.979959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.980026 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.980031 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:16:46.980093 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 20:16:46.980136 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":31,"thread_start_us":94,"threads_started":1}
I20260807 20:16:46.980225 24732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.975097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60014 (local address 127.23.200.195:40341)
0807 20:16:46.975284 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.975288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.975489 (+ 201us) server_negotiation.cc:407] Connection header received
0807 20:16:46.975533 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.975535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.975549 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.975624 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:46.975815 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.976864 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.979149 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.979876 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.979958 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.980051 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.980110 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.980147 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":76,"threads_started":1}
I20260807 20:16:46.980320 24161 tablet_service.cc:1467] Processing CreateTablet for tablet e222b94854674765bd7f47cc75a27db1 (DEFAULT_TABLE table=test-table [id=1066b16ba2024c909dfcf2dd3fd5118b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:46.980717 24161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e222b94854674765bd7f47cc75a27db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:46.980950 24604 tablet_service.cc:1467] Processing CreateTablet for tablet e222b94854674765bd7f47cc75a27db1 (DEFAULT_TABLE table=test-table [id=1066b16ba2024c909dfcf2dd3fd5118b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:46.981267 24604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e222b94854674765bd7f47cc75a27db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:46.981465 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.974862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.194:33219 (local address 127.0.0.1:49580)
0807 20:16:46.975252 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:46.975260 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:46.975297 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:46.975695 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:46.975706 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:46.975708 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:46.975735 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.975736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.977879 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.977880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.978838 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:46.978841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.981207 (+ 2366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.981208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:46.981286 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.981290 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:16:46.981348 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 20:16:46.981397 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":2,"thread_start_us":109,"threads_started":1}
I20260807 20:16:46.981684 24730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:46.975044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49580 (local address 127.23.200.194:33219)
0807 20:16:46.975505 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:46.975509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:46.975517 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:16:46.975550 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:46.975552 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:46.975566 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:46.975634 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:46.976944 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.977809 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.978937 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:46.980250 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:46.980590 (+ 340us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:46.980681 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:46.981567 (+ 886us) server_negotiation.cc:299] Negotiation successful
0807 20:16:46.981609 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":103,"threads_started":1}
I20260807 20:16:46.983560 24733 tablet_bootstrap.cc:492] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2: Bootstrap starting.
I20260807 20:16:46.983888 24734 tablet_bootstrap.cc:492] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5: Bootstrap starting.
I20260807 20:16:46.984395 24733 tablet_bootstrap.cc:654] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:46.984450 24734 tablet_bootstrap.cc:654] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:46.984458 24382 tablet_service.cc:1467] Processing CreateTablet for tablet e222b94854674765bd7f47cc75a27db1 (DEFAULT_TABLE table=test-table [id=1066b16ba2024c909dfcf2dd3fd5118b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:16:46.984720 24734 log.cc:824] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:46.984716 24733 log.cc:824] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:46.984809 24382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e222b94854674765bd7f47cc75a27db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:46.985489 24734 tablet_bootstrap.cc:492] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5: No bootstrap required, opened a new log
I20260807 20:16:46.985562 24734 ts_tablet_manager.cc:1397] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:16:46.985587 24733 tablet_bootstrap.cc:492] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2: No bootstrap required, opened a new log
I20260807 20:16:46.985652 24733 ts_tablet_manager.cc:1397] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:16:46.987556 24733 raft_consensus.cc:348] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33219 } } peers { permanent_uuid: "065e9036915f4100a9c1e200b07289c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45007 } } peers { permanent_uuid: "0e99ba920ed449898be2796236c9a2c5" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40341 } }
I20260807 20:16:46.987716 24733 raft_consensus.cc:374] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:46.987761 24733 raft_consensus.cc:729] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 065e9036915f4100a9c1e200b07289c2, State: Initialized, Role: FOLLOWER
I20260807 20:16:46.987883 24733 consensus_queue.cc:260] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33219 } } peers { permanent_uuid: "065e9036915f4100a9c1e200b07289c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45007 } } peers { permanent_uuid: "0e99ba920ed449898be2796236c9a2c5" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40341 } }
I20260807 20:16:46.988037 24734 raft_consensus.cc:348] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33219 } } peers { permanent_uuid: "065e9036915f4100a9c1e200b07289c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45007 } } peers { permanent_uuid: "0e99ba920ed449898be2796236c9a2c5" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40341 } }
I20260807 20:16:46.988147 24733 ts_tablet_manager.cc:1428] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 20:16:46.988168 24268 heartbeater.cc:503] Master 127.23.200.254:33203 was elected leader, sending a full tablet report...
I20260807 20:16:46.988195 24734 raft_consensus.cc:374] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:46.988233 24734 raft_consensus.cc:729] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e99ba920ed449898be2796236c9a2c5, State: Initialized, Role: FOLLOWER
I20260807 20:16:46.988340 24734 consensus_queue.cc:260] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33219 } } peers { permanent_uuid: "065e9036915f4100a9c1e200b07289c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45007 } } peers { permanent_uuid: "0e99ba920ed449898be2796236c9a2c5" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40341 } }
I20260807 20:16:46.988591 24734 ts_tablet_manager.cc:1428] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:16:46.988622 24703 heartbeater.cc:503] Master 127.23.200.254:33203 was elected leader, sending a full tablet report...
I20260807 20:16:46.988792 24737 tablet_bootstrap.cc:492] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a: Bootstrap starting.
I20260807 20:16:46.989476 24737 tablet_bootstrap.cc:654] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:46.989787 24737 log.cc:824] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:46.990495 24737 tablet_bootstrap.cc:492] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a: No bootstrap required, opened a new log
I20260807 20:16:46.990562 24737 ts_tablet_manager.cc:1397] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 20:16:46.992308 24737 raft_consensus.cc:348] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33219 } } peers { permanent_uuid: "065e9036915f4100a9c1e200b07289c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45007 } } peers { permanent_uuid: "0e99ba920ed449898be2796236c9a2c5" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40341 } }
I20260807 20:16:46.992430 24737 raft_consensus.cc:374] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:46.992460 24737 raft_consensus.cc:729] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d71d7abc67f4fd68a7fc59b1fa7fc6a, State: Initialized, Role: FOLLOWER
I20260807 20:16:46.992566 24737 consensus_queue.cc:260] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33219 } } peers { permanent_uuid: "065e9036915f4100a9c1e200b07289c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45007 } } peers { permanent_uuid: "0e99ba920ed449898be2796236c9a2c5" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40341 } }
I20260807 20:16:46.992812 24737 ts_tablet_manager.cc:1428] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 20:16:46.992900 24489 heartbeater.cc:503] Master 127.23.200.254:33203 was elected leader, sending a full tablet report...
I20260807 20:16:47.018131 24742 raft_consensus.cc:482] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:16:47.018275 24742 raft_consensus.cc:504] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33219 } } peers { permanent_uuid: "065e9036915f4100a9c1e200b07289c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45007 } } peers { permanent_uuid: "0e99ba920ed449898be2796236c9a2c5" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40341 } }
I20260807 20:16:47.018710 24742 leader_election.cc:288] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 065e9036915f4100a9c1e200b07289c2 (127.23.200.193:45007), 0e99ba920ed449898be2796236c9a2c5 (127.23.200.195:40341)
I20260807 20:16:47.022573 24732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:47.019066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:33077 (local address 127.23.200.195:40341)
0807 20:16:47.019113 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:47.019115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:47.019220 (+ 105us) server_negotiation.cc:407] Connection header received
0807 20:16:47.019262 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:47.019262 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:47.019270 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:47.019316 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:47.019430 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:47.020110 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:47.021645 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:47.022214 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:47.022294 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:47.022303 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:47.022458 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 20:16:47.022501 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:16:47.022575 24728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:47.018979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.200.194:45829 (local address 127.23.200.193:45007)
0807 20:16:47.019040 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:47.019043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:47.019053 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:16:47.019083 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:47.019084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:47.019097 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:47.019149 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:16:47.019323 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:47.020080 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:47.021645 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:47.022210 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:47.022294 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:47.022304 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:47.022478 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 20:16:47.022516 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:16:47.022596 24495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:47.018902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:45007 (local address 127.23.200.194:45829)
0807 20:16:47.018953 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:47.018965 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:47.019012 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:47.019168 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:47.019169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:47.019171 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:47.019246 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:47.019248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:47.020189 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:47.020191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:47.021539 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:47.021543 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:47.022297 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:47.022298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:47.022412 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:47.022415 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:16:47.022465 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 20:16:47.022496 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":25}
I20260807 20:16:47.022603 24744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:47.018910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.195:40341 (local address 127.23.200.194:33077)
0807 20:16:47.019153 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:47.019161 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:16:47.019213 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:16:47.019326 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:16:47.019326 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:16:47.019328 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:16:47.019379 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:47.019381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:47.020189 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:47.020191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:47.021544 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:16:47.021545 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:16:47.022295 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:16:47.022296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:16:47.022383 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:47.022389 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 20:16:47.022450 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 20:16:47.022492 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":175,"thread_start_us":122,"threads_started":1}
I20260807 20:16:47.023170 24637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e222b94854674765bd7f47cc75a27db1" candidate_uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e99ba920ed449898be2796236c9a2c5" is_pre_election: true
I20260807 20:16:47.023232 24194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e222b94854674765bd7f47cc75a27db1" candidate_uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "065e9036915f4100a9c1e200b07289c2" is_pre_election: true
I20260807 20:16:47.023324 24637 raft_consensus.cc:2454] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d71d7abc67f4fd68a7fc59b1fa7fc6a in term 0.
I20260807 20:16:47.023391 24194 raft_consensus.cc:2454] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d71d7abc67f4fd68a7fc59b1fa7fc6a in term 0.
I20260807 20:16:47.023564 24296 leader_election.cc:302] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e99ba920ed449898be2796236c9a2c5, 2d71d7abc67f4fd68a7fc59b1fa7fc6a; no voters:
I20260807 20:16:47.023705 24742 raft_consensus.cc:2781] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:16:47.023757 24742 raft_consensus.cc:482] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:16:47.023779 24742 raft_consensus.cc:3037] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:47.024459 24742 raft_consensus.cc:504] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33219 } } peers { permanent_uuid: "065e9036915f4100a9c1e200b07289c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45007 } } peers { permanent_uuid: "0e99ba920ed449898be2796236c9a2c5" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40341 } }
I20260807 20:16:47.024639 24742 leader_election.cc:288] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [CANDIDATE]: Term 1 election: Requested vote from peers 065e9036915f4100a9c1e200b07289c2 (127.23.200.193:45007), 0e99ba920ed449898be2796236c9a2c5 (127.23.200.195:40341)
I20260807 20:16:47.024721 24637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e222b94854674765bd7f47cc75a27db1" candidate_uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e99ba920ed449898be2796236c9a2c5"
I20260807 20:16:47.024729 24194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e222b94854674765bd7f47cc75a27db1" candidate_uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "065e9036915f4100a9c1e200b07289c2"
I20260807 20:16:47.024792 24637 raft_consensus.cc:3037] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:47.024819 24194 raft_consensus.cc:3037] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:47.025528 24637 raft_consensus.cc:2454] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d71d7abc67f4fd68a7fc59b1fa7fc6a in term 1.
I20260807 20:16:47.025632 24194 raft_consensus.cc:2454] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d71d7abc67f4fd68a7fc59b1fa7fc6a in term 1.
I20260807 20:16:47.025691 24296 leader_election.cc:302] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e99ba920ed449898be2796236c9a2c5, 2d71d7abc67f4fd68a7fc59b1fa7fc6a; no voters:
I20260807 20:16:47.025768 24742 raft_consensus.cc:2781] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:47.025910 24742 raft_consensus.cc:686] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [term 1 LEADER]: Becoming Leader. State: Replica: 2d71d7abc67f4fd68a7fc59b1fa7fc6a, State: Running, Role: LEADER
I20260807 20:16:47.026001 24742 consensus_queue.cc:237] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33219 } } peers { permanent_uuid: "065e9036915f4100a9c1e200b07289c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45007 } } peers { permanent_uuid: "0e99ba920ed449898be2796236c9a2c5" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40341 } }
I20260807 20:16:47.026908 23949 catalog_manager.cc:5495] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d71d7abc67f4fd68a7fc59b1fa7fc6a (127.23.200.194). New cstate: current_term: 1 leader_uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d71d7abc67f4fd68a7fc59b1fa7fc6a" member_type: VOTER last_known_addr { host: "127.23.200.194" port: 33219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "065e9036915f4100a9c1e200b07289c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e99ba920ed449898be2796236c9a2c5" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40341 } health_report { overall_health: UNKNOWN } } }
I20260807 20:16:47.040112 24730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:47.037787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49592 (local address 127.23.200.194:33219)
0807 20:16:47.037861 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:47.037863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:47.037873 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:16:47.037936 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:47.037937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:47.037953 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:47.038016 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:16:47.038126 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:47.038622 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:47.039379 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:47.039582 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:47.039627 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:47.039824 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:16:47.039931 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:16:47.039984 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:47.040015 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 20:16:47.040030 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 20:16:47.042073 24194 raft_consensus.cc:1261] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2 [term 1 FOLLOWER]: Refusing update from remote peer 2d71d7abc67f4fd68a7fc59b1fa7fc6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:16:47.042073 24637 raft_consensus.cc:1261] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5 [term 1 FOLLOWER]: Refusing update from remote peer 2d71d7abc67f4fd68a7fc59b1fa7fc6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:16:47.042492 24745 consensus_queue.cc:1035] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e99ba920ed449898be2796236c9a2c5" member_type: VOTER last_known_addr { host: "127.23.200.195" port: 40341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:16:47.042629 24742 consensus_queue.cc:1035] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "065e9036915f4100a9c1e200b07289c2" member_type: VOTER last_known_addr { host: "127.23.200.193" port: 45007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:16:47.045327 24755 mvcc.cc:205] Tried to move back new op lower bound from 7316004073641697280 to 7316004073579225088. Current Snapshot: MvccSnapshot[applied={T|T < 7316004073641697280}]
I20260807 20:16:47.046038 24756 mvcc.cc:205] Tried to move back new op lower bound from 7316004073641697280 to 7316004073579225088. Current Snapshot: MvccSnapshot[applied={T|T < 7316004073641697280}]
I20260807 20:16:47.047246 23949 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:16:47.047338 23949 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:16:47.048022 23949 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4bbeea346c7841aa8d816a1a5e09b39f: Sending DeleteTablet for 3 replicas of tablet e222b94854674765bd7f47cc75a27db1
I20260807 20:16:47.048310 24382 tablet_service.cc:1514] Processing DeleteTablet for tablet e222b94854674765bd7f47cc75a27db1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:47 UTC) from {username='slave'} at 127.0.0.1:49580
I20260807 20:16:47.048316 24161 tablet_service.cc:1514] Processing DeleteTablet for tablet e222b94854674765bd7f47cc75a27db1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:47 UTC) from {username='slave'} at 127.0.0.1:35024
I20260807 20:16:47.048537 24773 tablet_replica.cc:330] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a: stopping tablet replica
I20260807 20:16:47.048535 24604 tablet_service.cc:1514] Processing DeleteTablet for tablet e222b94854674765bd7f47cc75a27db1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:16:47 UTC) from {username='slave'} at 127.0.0.1:60014
I20260807 20:16:47.048542 24775 tablet_replica.cc:330] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2: stopping tablet replica
I20260807 20:16:47.048640 24773 raft_consensus.cc:2229] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:16:47.048671 24775 raft_consensus.cc:2229] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:16:47.048740 24773 raft_consensus.cc:2258] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:47.048828 24775 raft_consensus.cc:2258] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:47.049185 24773 ts_tablet_manager.cc:1905] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:16:47.049213 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 24055
I20260807 20:16:47.049355 24775 ts_tablet_manager.cc:1905] T e222b94854674765bd7f47cc75a27db1 P 065e9036915f4100a9c1e200b07289c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:16:47.051160 24773 ts_tablet_manager.cc:1918] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:16:47.051249 24773 log.cc:1196] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-1/wal/wals/e222b94854674765bd7f47cc75a27db1
I20260807 20:16:47.051551 24773 ts_tablet_manager.cc:1939] T e222b94854674765bd7f47cc75a27db1 P 2d71d7abc67f4fd68a7fc59b1fa7fc6a: Deleting consensus metadata
I20260807 20:16:47.052011 23916 catalog_manager.cc:4841] TS 2d71d7abc67f4fd68a7fc59b1fa7fc6a (127.23.200.194:33219): tablet e222b94854674765bd7f47cc75a27db1 (table test-table [id=1066b16ba2024c909dfcf2dd3fd5118b]) successfully deleted
W20260807 20:16:47.057868 23918 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.200.193:45007 (error 108)
I20260807 20:16:47.057940 24776 tablet_replica.cc:330] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5: stopping tablet replica
I20260807 20:16:47.058069 24776 raft_consensus.cc:2229] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:16:47.058208 24776 raft_consensus.cc:2258] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:16:47.058573 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:47.058426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.193:45007 (local address 127.0.0.1:35040)
0807 20:16:47.058482 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 20:16:47.058535 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.193:45007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 20:16:47.058676 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 24274
I20260807 20:16:47.076840 24776 ts_tablet_manager.cc:1905] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 20:16:47.077944 23918 catalog_manager.cc:4584] TS 065e9036915f4100a9c1e200b07289c2 (127.23.200.193:45007): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e222b94854674765bd7f47cc75a27db1: Network error: Client connection negotiation failed: client connection to 127.23.200.193:45007: connect: Connection refused (error 111)
I20260807 20:16:47.078742 24776 ts_tablet_manager.cc:1918] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:16:47.078840 24776 log.cc:1196] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5: Deleting WAL directory at /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133562076833-24355-0/minicluster-data/ts-2/wal/wals/e222b94854674765bd7f47cc75a27db1
I20260807 20:16:47.079103 24776 ts_tablet_manager.cc:1939] T e222b94854674765bd7f47cc75a27db1 P 0e99ba920ed449898be2796236c9a2c5: Deleting consensus metadata
I20260807 20:16:47.079711 23919 catalog_manager.cc:4841] TS 0e99ba920ed449898be2796236c9a2c5 (127.23.200.195:40341): tablet e222b94854674765bd7f47cc75a27db1 (table test-table [id=1066b16ba2024c909dfcf2dd3fd5118b]) successfully deleted
I20260807 20:16:47.083518 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 24498
I20260807 20:16:47.105417 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 23886
2026-08-07T20:16:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (698 ms)
[----------] 6 tests from EncryptionPolicyTest (8696 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:16:47 dist-test-slave-2j7r krb5kdc[24783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:16:47 dist-test-slave-2j7r krb5kdc[24783](info): set up 2 sockets
Aug 07 20:16:47 dist-test-slave-2j7r krb5kdc[24783](info): commencing operation
krb5kdc: starting...
W20260807 20:16:48.268898 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:16:48 dist-test-slave-2j7r krb5kdc[24783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:16:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:16:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:16:48.389307 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41607
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:48.471437 24804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:48.471635 24804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:48.471655 24804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:48.473040 24804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:16:48.473119 24804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:48.473150 24804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:48.473178 24804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:16:48.473198 24804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:16:48.474820 24804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41607
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:48.475045 24804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:48.475266 24804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:48.480116 24814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:48.480173 24812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:48.480115 24810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:48.480163 24804 server_base.cc:1034] running on GCE node
I20260807 20:16:48.480505 24804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:48.480748 24804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:48.481889 24804 hybrid_clock.cc:648] HybridClock initialized: now 1786133808481873 us; error 28 us; skew 500 ppm
Aug 07 20:16:48 dist-test-slave-2j7r krb5kdc[24783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133808, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:48.484328 24804 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:16:48.484774 24804 webserver.cc:460] Webserver started at http://127.0.0.1:41993/ using document root <none> and password file <none>
I20260807 20:16:48.484970 24804 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:48.485018 24804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:48.485162 24804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:48.486210 24804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "585f4b245e6a43f1b56842dbc880ba21"
format_stamp: "Formatted at 2026-08-07 20:16:48 on dist-test-slave-2j7r"
server_key: "4bc508528a577a3d7d13821bfc1d658e"
server_key_iv: "97d74451b3d880d5484908a610d894bf"
server_key_version: "encryptionkey@0"
I20260807 20:16:48.486545 24804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "585f4b245e6a43f1b56842dbc880ba21"
format_stamp: "Formatted at 2026-08-07 20:16:48 on dist-test-slave-2j7r"
server_key: "4bc508528a577a3d7d13821bfc1d658e"
server_key_iv: "97d74451b3d880d5484908a610d894bf"
server_key_version: "encryptionkey@0"
I20260807 20:16:48.487773 24804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 20:16:48.488481 24823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:48.488649 24804 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:16:48.488735 24804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "585f4b245e6a43f1b56842dbc880ba21"
format_stamp: "Formatted at 2026-08-07 20:16:48 on dist-test-slave-2j7r"
server_key: "4bc508528a577a3d7d13821bfc1d658e"
server_key_iv: "97d74451b3d880d5484908a610d894bf"
server_key_version: "encryptionkey@0"
I20260807 20:16:48.488793 24804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:48.495064 24804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:48.496821 24804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:48.496949 24804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:48.500784 24804 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41607
I20260807 20:16:48.500819 24908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41607 every 8 connection(s)
I20260807 20:16:48.501197 24804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:16:48.502070 24909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:48.504211 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 24804
I20260807 20:16:48.504300 24909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21: Bootstrap starting.
I20260807 20:16:48.504319 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:16:48.504498 24355 external_mini_cluster.cc:1428] Setting key 61ef2278a07d50175739a831d6374fa4
I20260807 20:16:48.504957 24909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:48.505316 24909 log.cc:824] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:48.506084 24909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21: No bootstrap required, opened a new log
I20260807 20:16:48.508249 24909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585f4b245e6a43f1b56842dbc880ba21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41607 } }
I20260807 20:16:48.508417 24909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:48.508479 24909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 585f4b245e6a43f1b56842dbc880ba21, State: Initialized, Role: FOLLOWER
I20260807 20:16:48.508603 24909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585f4b245e6a43f1b56842dbc880ba21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41607 } }
I20260807 20:16:48.508680 24909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:16:48.508718 24909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:16:48.508776 24909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:48.509367 24909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585f4b245e6a43f1b56842dbc880ba21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41607 } }
I20260807 20:16:48.509481 24909 leader_election.cc:302] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 585f4b245e6a43f1b56842dbc880ba21; no voters:
I20260807 20:16:48.509639 24909 leader_election.cc:288] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:16:48.509792 24916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:48.509912 24909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:16:48.509914 24916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [term 1 LEADER]: Becoming Leader. State: Replica: 585f4b245e6a43f1b56842dbc880ba21, State: Running, Role: LEADER
I20260807 20:16:48.510027 24916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585f4b245e6a43f1b56842dbc880ba21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41607 } }
I20260807 20:16:48.510555 24917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 585f4b245e6a43f1b56842dbc880ba21. Latest consensus state: current_term: 1 leader_uuid: "585f4b245e6a43f1b56842dbc880ba21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585f4b245e6a43f1b56842dbc880ba21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41607 } } }
I20260807 20:16:48.510641 24917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:48.510810 24915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "585f4b245e6a43f1b56842dbc880ba21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585f4b245e6a43f1b56842dbc880ba21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41607 } } }
I20260807 20:16:48.510867 24915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:48.510979 24930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:16:48.511494 24930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 20:16:48 dist-test-slave-2j7r krb5kdc[24783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133808, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 20:16:48.513258 24930 catalog_manager.cc:1347] Generated new cluster ID: d43279c52a9d4928b46205f38740e251
I20260807 20:16:48.513312 24930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:16:48.515379 24913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:48.505035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46451 (local address 127.0.0.1:41607)
0807 20:16:48.505219 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:48.505227 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:48.505254 (+ 27us) server_negotiation.cc:407] Connection header received
0807 20:16:48.505817 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:48.505825 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:48.505877 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:48.506012 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:48.506859 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:48.507735 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:48.508742 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:48.509269 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:48.509353 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:48.512332 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:48.512337 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:48.512337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:48.512350 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:48.513961 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:48.514321 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:48.514324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:48.514324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:48.514404 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:48.514603 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:48.514604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:48.514604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:48.514827 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:48.514977 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:48.515108 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 20:16:48.515204 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260807 20:16:48.518044 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:16:48.526588 24913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:48.519586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59689 (local address 127.0.0.1:41607)
0807 20:16:48.519621 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:48.519622 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:48.521252 (+ 1630us) server_negotiation.cc:407] Connection header received
0807 20:16:48.521315 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:48.521316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:48.521327 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:48.521371 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:48.521856 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:48.522370 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:48.523026 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:48.523227 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:48.523282 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:48.524325 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:48.524327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:48.524327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:48.524334 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:48.525546 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:48.525866 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:48.525867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:48.525867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:48.525918 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:48.526118 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:48.526118 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:48.526119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:48.526218 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:48.526305 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:48.526448 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 20:16:48.526505 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:16:48.530655 24930 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:16:48.531147 24930 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:16:48.534842 24930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 585f4b245e6a43f1b56842dbc880ba21: Generated new TSK 0
I20260807 20:16:48.534998 24930 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:16:48.538694 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 24804
W20260807 20:16:48.549574 24780 connection.cc:535] client connection to 127.0.0.1:41607 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41607): Connection reset by peer (error 104)
2026-08-07T20:16:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:16:48 dist-test-slave-2j7r krb5kdc[24953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:16:48 dist-test-slave-2j7r krb5kdc[24953](info): set up 2 sockets
Aug 07 20:16:48 dist-test-slave-2j7r krb5kdc[24953](info): commencing operation
krb5kdc: starting...
W20260807 20:16:49.727497 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:16:49 dist-test-slave-2j7r krb5kdc[24953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:16:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:16:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:16:49.849301 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38171
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:49.931296 24976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:49.931509 24976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:49.931531 24976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:49.932638 24976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:16:49.932672 24976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:49.932685 24976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:49.932696 24976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:16:49.932708 24976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:16:49.934160 24976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38171
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:49.934363 24976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:49.934546 24976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:49.939760 24984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:49.939769 24987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:49.939760 24985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:49.939800 24976 server_base.cc:1034] running on GCE node
I20260807 20:16:49.940156 24976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:49.940469 24976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:49.941617 24976 hybrid_clock.cc:648] HybridClock initialized: now 1786133809941606 us; error 30 us; skew 500 ppm
Aug 07 20:16:49 dist-test-slave-2j7r krb5kdc[24953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133809, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:49.944029 24976 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:16:49.944476 24976 webserver.cc:460] Webserver started at http://127.0.0.1:36043/ using document root <none> and password file <none>
I20260807 20:16:49.944671 24976 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:49.944717 24976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:49.944850 24976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:49.945933 24976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "e54524dfe63b4572a331a029119e8f59"
format_stamp: "Formatted at 2026-08-07 20:16:49 on dist-test-slave-2j7r"
server_key: "8508d826b1efa0513a64e0dcf4041bd8"
server_key_iv: "799a202159fea55432ab91d8c196ef77"
server_key_version: "encryptionkey@0"
I20260807 20:16:49.946290 24976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "e54524dfe63b4572a331a029119e8f59"
format_stamp: "Formatted at 2026-08-07 20:16:49 on dist-test-slave-2j7r"
server_key: "8508d826b1efa0513a64e0dcf4041bd8"
server_key_iv: "799a202159fea55432ab91d8c196ef77"
server_key_version: "encryptionkey@0"
I20260807 20:16:49.947554 24976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 20:16:49.948251 24995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:49.948405 24976 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:16:49.948493 24976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "e54524dfe63b4572a331a029119e8f59"
format_stamp: "Formatted at 2026-08-07 20:16:49 on dist-test-slave-2j7r"
server_key: "8508d826b1efa0513a64e0dcf4041bd8"
server_key_iv: "799a202159fea55432ab91d8c196ef77"
server_key_version: "encryptionkey@0"
I20260807 20:16:49.948560 24976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:49.958667 24976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:49.960505 24976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:49.960656 24976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:49.964711 24976 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38171
I20260807 20:16:49.964742 25072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38171 every 8 connection(s)
I20260807 20:16:49.965147 24976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:16:49.966032 25073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:49.968315 25073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59: Bootstrap starting.
I20260807 20:16:49.968894 25073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:49.969208 25073 log.cc:824] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:49.969868 25073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59: No bootstrap required, opened a new log
I20260807 20:16:49.971426 25073 raft_consensus.cc:348] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54524dfe63b4572a331a029119e8f59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38171 } }
I20260807 20:16:49.971561 25073 raft_consensus.cc:374] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:49.971585 25073 raft_consensus.cc:729] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54524dfe63b4572a331a029119e8f59, State: Initialized, Role: FOLLOWER
I20260807 20:16:49.971727 25073 consensus_queue.cc:260] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54524dfe63b4572a331a029119e8f59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38171 } }
I20260807 20:16:49.971801 25073 raft_consensus.cc:388] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:16:49.971827 25073 raft_consensus.cc:482] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:16:49.971872 25073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:49.972373 25073 raft_consensus.cc:504] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54524dfe63b4572a331a029119e8f59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38171 } }
I20260807 20:16:49.972467 25073 leader_election.cc:302] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e54524dfe63b4572a331a029119e8f59; no voters:
I20260807 20:16:49.972620 25073 leader_election.cc:288] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:16:49.972656 25077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:49.972801 25077 raft_consensus.cc:686] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [term 1 LEADER]: Becoming Leader. State: Replica: e54524dfe63b4572a331a029119e8f59, State: Running, Role: LEADER
I20260807 20:16:49.972873 25073 sys_catalog.cc:564] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:16:49.972893 25077 consensus_queue.cc:237] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54524dfe63b4572a331a029119e8f59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38171 } }
I20260807 20:16:49.973353 25082 sys_catalog.cc:455] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e54524dfe63b4572a331a029119e8f59. Latest consensus state: current_term: 1 leader_uuid: "e54524dfe63b4572a331a029119e8f59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54524dfe63b4572a331a029119e8f59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38171 } } }
I20260807 20:16:49.973438 25082 sys_catalog.cc:458] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:49.973565 25081 sys_catalog.cc:455] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e54524dfe63b4572a331a029119e8f59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54524dfe63b4572a331a029119e8f59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38171 } } }
I20260807 20:16:49.973677 25081 sys_catalog.cc:458] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:49.973903 25090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:16:49.974334 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 24976
I20260807 20:16:49.974404 25090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:16:49.974464 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:16:49.974675 24355 external_mini_cluster.cc:1428] Setting key af22f20c9bc58a7b104ecaf6de2e31f2
I20260807 20:16:49.975944 25090 catalog_manager.cc:1347] Generated new cluster ID: 7a4b63908e6f4ce98d794d2209ec1d7e
I20260807 20:16:49.975996 25090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:16:49.979550 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:49.975100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34077 (local address 127.0.0.1:38171)
0807 20:16:49.975267 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:49.975276 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:49.975374 (+ 98us) server_negotiation.cc:407] Connection header received
0807 20:16:49.975542 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:49.975550 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:49.975593 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:49.975714 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:49.975949 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:49.976874 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:49.977697 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:49.978181 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:49.978279 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:49.978940 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:49.978944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:49.978944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:49.979109 (+ 165us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:49.979140 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:49.979150 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:49.979220 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:49.979313 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 20:16:49.979343 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260807 20:16:49.980443 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:16:49.985642 25090 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:16:49.986158 25090 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:16:49.988765 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:49.981791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37273 (local address 127.0.0.1:38171)
0807 20:16:49.981862 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:49.981864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:49.985301 (+ 3437us) server_negotiation.cc:407] Connection header received
0807 20:16:49.985475 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:49.985476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:49.985487 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:49.985554 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:49.985703 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:49.986480 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:49.987471 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:49.987801 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:49.987863 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:49.988527 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:49.988529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:49.988529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:49.988534 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:49.988551 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:49.988555 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:49.988615 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:49.988672 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 20:16:49.988686 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 20:16:49.990329 25090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e54524dfe63b4572a331a029119e8f59: Generated new TSK 0
I20260807 20:16:49.990509 25090 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:16:49.995326 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 24976
W20260807 20:16:50.026417 24950 connection.cc:535] client connection to 127.0.0.1:38171 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38171): Connection reset by peer (error 104)
2026-08-07T20:16:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1476 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:16:50 dist-test-slave-2j7r krb5kdc[25124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:16:50 dist-test-slave-2j7r krb5kdc[25124](info): set up 2 sockets
Aug 07 20:16:50 dist-test-slave-2j7r krb5kdc[25124](info): commencing operation
krb5kdc: starting...
W20260807 20:16:51.202396 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:16:51 dist-test-slave-2j7r krb5kdc[25124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:16:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:16:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:16:51.321722 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42509
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:51.404762 25143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:51.404983 25143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:51.405014 25143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:51.406230 25143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:16:51.406277 25143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:51.406297 25143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:51.406313 25143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:16:51.406329 25143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:16:51.407847 25143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42509
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:51.408105 25143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:51.408324 25143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:51.413305 25157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:51.413337 25149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:51.413379 25153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:51.413574 25143 server_base.cc:1034] running on GCE node
I20260807 20:16:51.413810 25143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:51.414083 25143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:51.415233 25143 hybrid_clock.cc:648] HybridClock initialized: now 1786133811415222 us; error 32 us; skew 500 ppm
Aug 07 20:16:51 dist-test-slave-2j7r krb5kdc[25124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133811, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:51.417825 25143 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:16:51.418280 25143 webserver.cc:460] Webserver started at http://127.0.0.1:37039/ using document root <none> and password file <none>
I20260807 20:16:51.418534 25143 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:51.418586 25143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:51.418710 25143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:51.419781 25143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "408216560f0a4a4dba97e01bd84246c2"
format_stamp: "Formatted at 2026-08-07 20:16:51 on dist-test-slave-2j7r"
server_key: "16b86d46b838c19f08fc6c10f5f321fc"
server_key_iv: "0323fa8fa5bcaee7d89257662327c190"
server_key_version: "encryptionkey@0"
I20260807 20:16:51.420114 25143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "408216560f0a4a4dba97e01bd84246c2"
format_stamp: "Formatted at 2026-08-07 20:16:51 on dist-test-slave-2j7r"
server_key: "16b86d46b838c19f08fc6c10f5f321fc"
server_key_iv: "0323fa8fa5bcaee7d89257662327c190"
server_key_version: "encryptionkey@0"
I20260807 20:16:51.421485 25143 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:16:51.422245 25171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:51.422444 25143 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:16:51.422534 25143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "408216560f0a4a4dba97e01bd84246c2"
format_stamp: "Formatted at 2026-08-07 20:16:51 on dist-test-slave-2j7r"
server_key: "16b86d46b838c19f08fc6c10f5f321fc"
server_key_iv: "0323fa8fa5bcaee7d89257662327c190"
server_key_version: "encryptionkey@0"
I20260807 20:16:51.422595 25143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:51.426203 25143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:51.428141 25143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:51.428277 25143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:51.432226 25143 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42509
I20260807 20:16:51.432271 25247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42509 every 8 connection(s)
I20260807 20:16:51.432595 25143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:16:51.433661 25248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:51.435930 25248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2: Bootstrap starting.
I20260807 20:16:51.436474 25248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:51.436718 25248 log.cc:824] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:51.437002 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 25143
I20260807 20:16:51.437153 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:16:51.437327 24355 external_mini_cluster.cc:1428] Setting key 3c92476c9212ebb522d6463adfd90bd6
I20260807 20:16:51.437582 25248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2: No bootstrap required, opened a new log
I20260807 20:16:51.439693 25248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408216560f0a4a4dba97e01bd84246c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } }
I20260807 20:16:51.439919 25248 raft_consensus.cc:374] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:51.439970 25248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408216560f0a4a4dba97e01bd84246c2, State: Initialized, Role: FOLLOWER
I20260807 20:16:51.440111 25248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408216560f0a4a4dba97e01bd84246c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } }
I20260807 20:16:51.440207 25248 raft_consensus.cc:388] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:16:51.440244 25248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:16:51.440286 25253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:51.437864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42417 (local address 127.0.0.1:42509)
0807 20:16:51.438062 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:51.438073 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:51.438108 (+ 35us) server_negotiation.cc:407] Connection header received
0807 20:16:51.438860 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:51.438870 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:51.438929 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:51.439083 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:51.439669 (+ 586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:51.439673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:51.439673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:51.439678 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:51.439721 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:51.439734 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:51.439808 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:51.439844 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 20:16:51.439886 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
I20260807 20:16:51.440294 25248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:51.441021 25248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408216560f0a4a4dba97e01bd84246c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } }
I20260807 20:16:51.441146 25248 leader_election.cc:302] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 408216560f0a4a4dba97e01bd84246c2; no voters:
I20260807 20:16:51.441305 25248 leader_election.cc:288] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:16:51.441336 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:16:51.441342 25254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:51.441501 25254 raft_consensus.cc:686] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [term 1 LEADER]: Becoming Leader. State: Replica: 408216560f0a4a4dba97e01bd84246c2, State: Running, Role: LEADER
I20260807 20:16:51.441541 25248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:16:51.441640 25254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408216560f0a4a4dba97e01bd84246c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } }
I20260807 20:16:51.442106 25259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 408216560f0a4a4dba97e01bd84246c2. Latest consensus state: current_term: 1 leader_uuid: "408216560f0a4a4dba97e01bd84246c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408216560f0a4a4dba97e01bd84246c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } } }
I20260807 20:16:51.442113 25255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "408216560f0a4a4dba97e01bd84246c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408216560f0a4a4dba97e01bd84246c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } } }
I20260807 20:16:51.442214 25259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:51.442216 25255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:51.442937 25269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:16:51.443452 25269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:16:51.444399 25253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:51.443114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48355 (local address 127.0.0.1:42509)
0807 20:16:51.443143 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:51.443144 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:51.443451 (+ 307us) server_negotiation.cc:407] Connection header received
0807 20:16:51.443498 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:51.443499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:51.443504 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:51.443560 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:51.444207 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:51.444208 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:51.444209 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:51.444213 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:51.444226 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:51.444229 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:51.444276 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:51.444340 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 20:16:51.444350 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260807 20:16:51.445066 25269 catalog_manager.cc:1347] Generated new cluster ID: e5550eae18e74d4ca9a300ecbc76ac09
I20260807 20:16:51.445134 25269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:16:51.449826 25269 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:16:51.449996 25269 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:16:51.453428 25269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 408216560f0a4a4dba97e01bd84246c2: Generated new TSK 0
I20260807 20:16:51.453584 25269 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:16:51.460300 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 25143
2026-08-07T20:16:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1455 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:16:51 dist-test-slave-2j7r krb5kdc[25292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:16:51 dist-test-slave-2j7r krb5kdc[25292](info): set up 2 sockets
Aug 07 20:16:51 dist-test-slave-2j7r krb5kdc[25292](info): commencing operation
krb5kdc: starting...
W20260807 20:16:52.626019 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:16:52 dist-test-slave-2j7r krb5kdc[25292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:16:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:16:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:16:52.748350 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33515
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:52.831954 25311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:52.832171 25311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:52.832201 25311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:52.833324 25311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:16:52.833379 25311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:52.833400 25311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:52.833415 25311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:16:52.833432 25311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:16:52.834872 25311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:52.835111 25311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:52.835318 25311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:52.840068 25324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:52.840051 25320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:52.840046 25319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:52.840147 25311 server_base.cc:1034] running on GCE node
I20260807 20:16:52.840476 25311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:52.840731 25311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:52.841889 25311 hybrid_clock.cc:648] HybridClock initialized: now 1786133812841869 us; error 32 us; skew 500 ppm
Aug 07 20:16:52 dist-test-slave-2j7r krb5kdc[25292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133812, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:52.844333 25311 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:16:52.844780 25311 webserver.cc:460] Webserver started at http://127.0.0.1:40821/ using document root <none> and password file <none>
I20260807 20:16:52.844977 25311 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:52.845021 25311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:52.845171 25311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:52.846231 25311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "faab25a786e7424db15ad115d7d243dc"
format_stamp: "Formatted at 2026-08-07 20:16:52 on dist-test-slave-2j7r"
server_key: "12650b6bd1c9e8bd11190db4abede100"
server_key_iv: "058b73de7ed9d120d2b62a7801597c12"
server_key_version: "encryptionkey@0"
I20260807 20:16:52.846572 25311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "faab25a786e7424db15ad115d7d243dc"
format_stamp: "Formatted at 2026-08-07 20:16:52 on dist-test-slave-2j7r"
server_key: "12650b6bd1c9e8bd11190db4abede100"
server_key_iv: "058b73de7ed9d120d2b62a7801597c12"
server_key_version: "encryptionkey@0"
I20260807 20:16:52.847824 25311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:16:52.848484 25333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:52.848639 25311 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:16:52.848721 25311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "faab25a786e7424db15ad115d7d243dc"
format_stamp: "Formatted at 2026-08-07 20:16:52 on dist-test-slave-2j7r"
server_key: "12650b6bd1c9e8bd11190db4abede100"
server_key_iv: "058b73de7ed9d120d2b62a7801597c12"
server_key_version: "encryptionkey@0"
I20260807 20:16:52.848788 25311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:52.861517 25311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:52.863845 25311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:52.863983 25311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:52.868361 25311 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33515
I20260807 20:16:52.868474 25411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33515 every 8 connection(s)
I20260807 20:16:52.868865 25311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:16:52.869930 25413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:52.872488 25413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc: Bootstrap starting.
I20260807 20:16:52.873072 25413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:52.873365 25413 log.cc:824] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:52.873590 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 25311
I20260807 20:16:52.873696 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:16:52.873858 24355 external_mini_cluster.cc:1428] Setting key 384f2141fbe3c2973b33279e81c7cb2a
I20260807 20:16:52.874087 25413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc: No bootstrap required, opened a new log
I20260807 20:16:52.876183 25413 raft_consensus.cc:348] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faab25a786e7424db15ad115d7d243dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33515 } }
I20260807 20:16:52.876358 25413 raft_consensus.cc:374] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:52.876406 25413 raft_consensus.cc:729] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faab25a786e7424db15ad115d7d243dc, State: Initialized, Role: FOLLOWER
I20260807 20:16:52.876531 25413 consensus_queue.cc:260] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faab25a786e7424db15ad115d7d243dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33515 } }
I20260807 20:16:52.876609 25413 raft_consensus.cc:388] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:16:52.876648 25413 raft_consensus.cc:482] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:16:52.876693 25413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:52.877391 25413 raft_consensus.cc:504] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faab25a786e7424db15ad115d7d243dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33515 } }
I20260807 20:16:52.877499 25413 leader_election.cc:302] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: faab25a786e7424db15ad115d7d243dc; no voters:
I20260807 20:16:52.877640 25413 leader_election.cc:288] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:16:52.877679 25420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:52.877815 25420 raft_consensus.cc:686] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [term 1 LEADER]: Becoming Leader. State: Replica: faab25a786e7424db15ad115d7d243dc, State: Running, Role: LEADER
I20260807 20:16:52.877908 25413 sys_catalog.cc:564] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:16:52.877954 25420 consensus_queue.cc:237] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faab25a786e7424db15ad115d7d243dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33515 } }
I20260807 20:16:52.878271 25420 sys_catalog.cc:455] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader faab25a786e7424db15ad115d7d243dc. Latest consensus state: current_term: 1 leader_uuid: "faab25a786e7424db15ad115d7d243dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faab25a786e7424db15ad115d7d243dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33515 } } }
I20260807 20:16:52.878333 25420 sys_catalog.cc:458] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:52.879120 25422 sys_catalog.cc:455] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "faab25a786e7424db15ad115d7d243dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faab25a786e7424db15ad115d7d243dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33515 } } }
I20260807 20:16:52.879181 25422 sys_catalog.cc:458] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc [sys.catalog]: This master's current role is: LEADER
W20260807 20:16:52.879845 25448 catalog_manager.cc:1558] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:16:52.879907 25448 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:16:52.880086 25427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:16:52.880288 25427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 20:16:52 dist-test-slave-2j7r krb5kdc[25292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 20:16:52.881844 25427 catalog_manager.cc:1347] Generated new cluster ID: b34224a969e9461b9332dbf85f9976df
I20260807 20:16:52.881894 25427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:16:52.885850 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:52.874543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37773 (local address 127.0.0.1:33515)
0807 20:16:52.874714 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:52.874724 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:52.874758 (+ 34us) server_negotiation.cc:407] Connection header received
0807 20:16:52.875413 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:52.875423 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:52.875489 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:52.875638 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:52.876239 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:52.877918 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:52.878903 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:52.879519 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:52.879630 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:52.882372 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:52.882376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:52.882377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:52.882388 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:52.884214 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:52.884581 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:52.884585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:52.884585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:52.884706 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:52.884883 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:52.884884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:52.884885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:52.885246 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:52.885435 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:52.885515 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 20:16:52.885655 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260807 20:16:52.886710 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:16:52.891608 25427 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:16:52.892071 25427 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:16:52.895632 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:52.888184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39391 (local address 127.0.0.1:33515)
0807 20:16:52.888217 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:52.888218 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:52.888340 (+ 122us) server_negotiation.cc:407] Connection header received
0807 20:16:52.888454 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:52.888455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:52.888466 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:52.888494 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:52.889206 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:52.889936 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:52.890955 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:52.891420 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:52.891464 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:52.892850 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:52.892852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:52.892852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:52.892859 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:52.894427 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:52.894803 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:52.894804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:52.894804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:52.894855 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:52.895038 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:52.895039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:52.895040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:52.895155 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:52.895221 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:52.895452 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 20:16:52.895549 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 20:16:52.895823 25427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P faab25a786e7424db15ad115d7d243dc: Generated new TSK 0
I20260807 20:16:52.895999 25427 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:16:52.902844 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 25311
2026-08-07T20:16:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1438 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:16:52 dist-test-slave-2j7r krb5kdc[25463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:16:52 dist-test-slave-2j7r krb5kdc[25463](info): set up 2 sockets
Aug 07 20:16:52 dist-test-slave-2j7r krb5kdc[25463](info): commencing operation
krb5kdc: starting...
W20260807 20:16:54.073032 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:16:54 dist-test-slave-2j7r krb5kdc[25463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:16:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:16:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:16:54.194056 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42761
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:54.278501 25483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:54.278702 25483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:54.278723 25483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:54.279881 25483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:16:54.279928 25483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:54.279942 25483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:54.279956 25483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:16:54.279980 25483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:16:54.281492 25483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:54.281725 25483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:54.281944 25483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:54.286798 25489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:54.286774 25490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:54.286818 25483 server_base.cc:1034] running on GCE node
W20260807 20:16:54.286760 25492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:54.287268 25483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:54.287578 25483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:54.288733 25483 hybrid_clock.cc:648] HybridClock initialized: now 1786133814288720 us; error 27 us; skew 500 ppm
Aug 07 20:16:54 dist-test-slave-2j7r krb5kdc[25463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133814, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:54.296280 25483 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:16:54.296768 25483 webserver.cc:460] Webserver started at http://127.0.0.1:43571/ using document root <none> and password file <none>
I20260807 20:16:54.296995 25483 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:54.297045 25483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:54.297197 25483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:54.298273 25483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "2e1639160fa944718381b3be20d7ecd0"
format_stamp: "Formatted at 2026-08-07 20:16:54 on dist-test-slave-2j7r"
server_key: "7967bf70f8a1bffaa79fcbe5ae2dc789"
server_key_iv: "72c5147240eba4836e15484c2722c134"
server_key_version: "encryptionkey@0"
I20260807 20:16:54.298601 25483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "2e1639160fa944718381b3be20d7ecd0"
format_stamp: "Formatted at 2026-08-07 20:16:54 on dist-test-slave-2j7r"
server_key: "7967bf70f8a1bffaa79fcbe5ae2dc789"
server_key_iv: "72c5147240eba4836e15484c2722c134"
server_key_version: "encryptionkey@0"
I20260807 20:16:54.299883 25483 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 20:16:54.300606 25501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:54.300770 25483 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:54.300848 25483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "2e1639160fa944718381b3be20d7ecd0"
format_stamp: "Formatted at 2026-08-07 20:16:54 on dist-test-slave-2j7r"
server_key: "7967bf70f8a1bffaa79fcbe5ae2dc789"
server_key_iv: "72c5147240eba4836e15484c2722c134"
server_key_version: "encryptionkey@0"
I20260807 20:16:54.300909 25483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:54.313527 25483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:54.315224 25483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:54.315358 25483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:54.319295 25483 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42761
I20260807 20:16:54.319348 25579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42761 every 8 connection(s)
I20260807 20:16:54.319680 25483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:16:54.320561 25580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:54.322918 25580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0: Bootstrap starting.
I20260807 20:16:54.323501 25580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:54.323779 25580 log.cc:824] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:54.324532 25580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0: No bootstrap required, opened a new log
I20260807 20:16:54.326165 25580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1639160fa944718381b3be20d7ecd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } }
I20260807 20:16:54.326306 25580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:54.326339 25580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e1639160fa944718381b3be20d7ecd0, State: Initialized, Role: FOLLOWER
I20260807 20:16:54.326489 25580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1639160fa944718381b3be20d7ecd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } }
I20260807 20:16:54.326553 25580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:16:54.326584 25580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:16:54.326634 25580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:54.327142 25580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1639160fa944718381b3be20d7ecd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } }
I20260807 20:16:54.327239 25580 leader_election.cc:302] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e1639160fa944718381b3be20d7ecd0; no voters:
I20260807 20:16:54.327410 25580 leader_election.cc:288] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:16:54.327502 25584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:54.327661 25584 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [term 1 LEADER]: Becoming Leader. State: Replica: 2e1639160fa944718381b3be20d7ecd0, State: Running, Role: LEADER
I20260807 20:16:54.327661 25580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:16:54.327775 25584 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1639160fa944718381b3be20d7ecd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } }
I20260807 20:16:54.328147 25589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e1639160fa944718381b3be20d7ecd0. Latest consensus state: current_term: 1 leader_uuid: "2e1639160fa944718381b3be20d7ecd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1639160fa944718381b3be20d7ecd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } } }
I20260807 20:16:54.328151 25586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e1639160fa944718381b3be20d7ecd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e1639160fa944718381b3be20d7ecd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42761 } } }
I20260807 20:16:54.328258 25586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:54.328428 25589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:54.328466 25592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:16:54.329142 25592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:16:54.329569 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 25483
I20260807 20:16:54.329691 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:16:54.329866 24355 external_mini_cluster.cc:1428] Setting key 534d955ad28b95d08db5e1cf8407eda3
I20260807 20:16:54.330775 25592 catalog_manager.cc:1347] Generated new cluster ID: 21b581c868cb4be8a680917e0ee18f19
I20260807 20:16:54.330832 25592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:16:54.336048 25624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:54.330328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48789 (local address 127.0.0.1:42761)
0807 20:16:54.330506 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:54.330515 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:54.330938 (+ 423us) server_negotiation.cc:407] Connection header received
0807 20:16:54.331091 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:54.331099 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:54.331158 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:54.331316 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:54.331471 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:54.332917 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:54.333946 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:54.334599 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:54.334706 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:54.335335 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:54.335339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:54.335340 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:54.335560 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:54.335601 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:54.335617 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:54.335687 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:54.335753 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 20:16:54.335823 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260807 20:16:54.337157 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:16:54.338063 25592 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:16:54.338752 25592 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:16:54.342089 25592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e1639160fa944718381b3be20d7ecd0: Generated new TSK 0
I20260807 20:16:54.342149 25624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:54.338678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48657 (local address 127.0.0.1:42761)
0807 20:16:54.338713 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:54.338715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:54.338787 (+ 72us) server_negotiation.cc:407] Connection header received
0807 20:16:54.338870 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:54.338871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:54.338882 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:54.338940 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:54.339096 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:54.339795 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:54.340843 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:54.341152 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:54.341203 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:54.341859 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:54.341861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:54.341861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:54.341866 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:54.341880 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:54.341885 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:54.341926 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:54.341993 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 20:16:54.342056 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:16:54.342299 25592 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:16:54.345500 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 25483
2026-08-07T20:16:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1444 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:16:54 dist-test-slave-2j7r krb5kdc[25637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:16:54 dist-test-slave-2j7r krb5kdc[25637](info): set up 2 sockets
Aug 07 20:16:54 dist-test-slave-2j7r krb5kdc[25637](info): commencing operation
krb5kdc: starting...
W20260807 20:16:55.536804 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:16:55 dist-test-slave-2j7r krb5kdc[25637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:16:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:16:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:16:55.659646 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36157
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:55.743906 25654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:55.744109 25654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:55.744139 25654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:55.745285 25654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:16:55.745325 25654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:55.745344 25654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:55.745360 25654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:16:55.745375 25654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:16:55.746840 25654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:55.747081 25654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:55.747287 25654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:55.751996 25662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:55.752032 25663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:55.752032 25665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:55.752094 25654 server_base.cc:1034] running on GCE node
I20260807 20:16:55.752439 25654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:55.752693 25654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:55.753826 25654 hybrid_clock.cc:648] HybridClock initialized: now 1786133815753807 us; error 32 us; skew 500 ppm
Aug 07 20:16:55 dist-test-slave-2j7r krb5kdc[25637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133815, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:55.756266 25654 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:16:55.756664 25654 webserver.cc:460] Webserver started at http://127.0.0.1:44891/ using document root <none> and password file <none>
I20260807 20:16:55.756896 25654 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:55.756953 25654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:55.757100 25654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:55.758121 25654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "cdde4f0d4eba48d48389c65d5c94b8b0"
format_stamp: "Formatted at 2026-08-07 20:16:55 on dist-test-slave-2j7r"
server_key: "b46f38d39c24efa9c878a6fc00aad8a1"
server_key_iv: "22db6a28f6272d86cb5aa6ed8c6eb5e3"
server_key_version: "encryptionkey@0"
I20260807 20:16:55.758450 25654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "cdde4f0d4eba48d48389c65d5c94b8b0"
format_stamp: "Formatted at 2026-08-07 20:16:55 on dist-test-slave-2j7r"
server_key: "b46f38d39c24efa9c878a6fc00aad8a1"
server_key_iv: "22db6a28f6272d86cb5aa6ed8c6eb5e3"
server_key_version: "encryptionkey@0"
I20260807 20:16:55.759686 25654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:16:55.760370 25675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:55.760527 25654 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:16:55.760612 25654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "cdde4f0d4eba48d48389c65d5c94b8b0"
format_stamp: "Formatted at 2026-08-07 20:16:55 on dist-test-slave-2j7r"
server_key: "b46f38d39c24efa9c878a6fc00aad8a1"
server_key_iv: "22db6a28f6272d86cb5aa6ed8c6eb5e3"
server_key_version: "encryptionkey@0"
I20260807 20:16:55.760673 25654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:55.763895 25654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:55.765679 25654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:55.765805 25654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:55.769572 25654 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36157
I20260807 20:16:55.769600 25749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36157 every 8 connection(s)
I20260807 20:16:55.769896 25654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:16:55.770856 25750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:55.773032 25750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0: Bootstrap starting.
I20260807 20:16:55.773598 25750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:55.773842 25750 log.cc:824] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:55.774580 25750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0: No bootstrap required, opened a new log
I20260807 20:16:55.774801 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 25654
I20260807 20:16:55.774915 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:16:55.775099 24355 external_mini_cluster.cc:1428] Setting key 9e4512f9b60ec583e2528cd62a80f28b
I20260807 20:16:55.776718 25750 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdde4f0d4eba48d48389c65d5c94b8b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36157 } }
I20260807 20:16:55.776862 25750 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:55.776885 25750 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdde4f0d4eba48d48389c65d5c94b8b0, State: Initialized, Role: FOLLOWER
I20260807 20:16:55.777034 25750 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdde4f0d4eba48d48389c65d5c94b8b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36157 } }
I20260807 20:16:55.777153 25750 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:16:55.777196 25750 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:16:55.777256 25750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:55.777583 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:55.775588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47175 (local address 127.0.0.1:36157)
0807 20:16:55.775806 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:55.775816 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:55.775849 (+ 33us) server_negotiation.cc:407] Connection header received
0807 20:16:55.776287 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:55.776296 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:55.776330 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:55.776483 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:55.777050 (+ 567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:55.777053 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:55.777054 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:55.777058 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:55.777139 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:55.777155 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:55.777225 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:55.777353 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 20:16:55.777403 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260807 20:16:55.778188 25750 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdde4f0d4eba48d48389c65d5c94b8b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36157 } }
I20260807 20:16:55.778303 25750 leader_election.cc:302] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdde4f0d4eba48d48389c65d5c94b8b0; no voters:
I20260807 20:16:55.778458 25750 leader_election.cc:288] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:16:55.778496 25762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:55.778497 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:16:55.778615 25762 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [term 1 LEADER]: Becoming Leader. State: Replica: cdde4f0d4eba48d48389c65d5c94b8b0, State: Running, Role: LEADER
I20260807 20:16:55.778744 25750 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:16:55.778759 25762 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdde4f0d4eba48d48389c65d5c94b8b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36157 } }
I20260807 20:16:55.779175 25766 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdde4f0d4eba48d48389c65d5c94b8b0. Latest consensus state: current_term: 1 leader_uuid: "cdde4f0d4eba48d48389c65d5c94b8b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdde4f0d4eba48d48389c65d5c94b8b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36157 } } }
I20260807 20:16:55.779258 25766 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:55.779470 25764 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdde4f0d4eba48d48389c65d5c94b8b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdde4f0d4eba48d48389c65d5c94b8b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36157 } } }
I20260807 20:16:55.779757 25764 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:55.779903 25774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:16:55.780375 25774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:16:55.782024 25774 catalog_manager.cc:1347] Generated new cluster ID: 7c39cf46f7324732ba93a3bda1b83475
I20260807 20:16:55.782104 25774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:16:55.782480 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:55.781368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54971 (local address 127.0.0.1:36157)
0807 20:16:55.781399 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:55.781401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:55.781522 (+ 121us) server_negotiation.cc:407] Connection header received
0807 20:16:55.781584 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:55.781585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:55.781591 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:55.781639 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:55.782273 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:55.782274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:55.782275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:55.782279 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:55.782295 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:55.782299 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:55.782355 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:55.782412 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 20:16:55.782423 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260807 20:16:55.787964 25774 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:16:55.788122 25774 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:16:55.792680 25774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdde4f0d4eba48d48389c65d5c94b8b0: Generated new TSK 0
I20260807 20:16:55.792872 25774 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:16:55.796860 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 25654
2026-08-07T20:16:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:16:55 dist-test-slave-2j7r krb5kdc[25804](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:16:55 dist-test-slave-2j7r krb5kdc[25804](info): set up 2 sockets
Aug 07 20:16:55 dist-test-slave-2j7r krb5kdc[25804](info): commencing operation
krb5kdc: starting...
W20260807 20:16:56.965309 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:16:57 dist-test-slave-2j7r krb5kdc[25804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:16:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:16:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:16:57.091791 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41531
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:57.175071 25823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:57.175272 25823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:57.175295 25823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:57.176378 25823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:16:57.176411 25823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:57.176424 25823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:57.176435 25823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:16:57.176445 25823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:16:57.177975 25823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:57.178172 25823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:57.178347 25823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:57.183145 25833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:57.183157 25835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:57.183233 25823 server_base.cc:1034] running on GCE node
W20260807 20:16:57.183158 25832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:57.183530 25823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:57.183827 25823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:57.184976 25823 hybrid_clock.cc:648] HybridClock initialized: now 1786133817184955 us; error 30 us; skew 500 ppm
Aug 07 20:16:57 dist-test-slave-2j7r krb5kdc[25804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133817, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:57.187409 25823 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:16:57.187846 25823 webserver.cc:460] Webserver started at http://127.0.0.1:45449/ using document root <none> and password file <none>
I20260807 20:16:57.188041 25823 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:57.188087 25823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:57.188210 25823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:57.189311 25823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "f0857e43488c493c8d9860ae70fc04f2"
format_stamp: "Formatted at 2026-08-07 20:16:57 on dist-test-slave-2j7r"
server_key: "e59af4191ff5255bd7a5fd2abd5100e5"
server_key_iv: "b54f8ffe214dc2135fc0a93d74be14fc"
server_key_version: "encryptionkey@0"
I20260807 20:16:57.189656 25823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "f0857e43488c493c8d9860ae70fc04f2"
format_stamp: "Formatted at 2026-08-07 20:16:57 on dist-test-slave-2j7r"
server_key: "e59af4191ff5255bd7a5fd2abd5100e5"
server_key_iv: "b54f8ffe214dc2135fc0a93d74be14fc"
server_key_version: "encryptionkey@0"
I20260807 20:16:57.190892 25823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:16:57.191524 25846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:57.191740 25823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:16:57.191872 25823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "f0857e43488c493c8d9860ae70fc04f2"
format_stamp: "Formatted at 2026-08-07 20:16:57 on dist-test-slave-2j7r"
server_key: "e59af4191ff5255bd7a5fd2abd5100e5"
server_key_iv: "b54f8ffe214dc2135fc0a93d74be14fc"
server_key_version: "encryptionkey@0"
I20260807 20:16:57.191936 25823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:57.214250 25823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:57.216346 25823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:57.216521 25823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:57.220626 25823 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41531
I20260807 20:16:57.220671 25922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41531 every 8 connection(s)
I20260807 20:16:57.221033 25823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:16:57.221923 25925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:57.224237 25925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2: Bootstrap starting.
I20260807 20:16:57.224828 25925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:57.225119 25925 log.cc:824] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:57.225824 25925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2: No bootstrap required, opened a new log
I20260807 20:16:57.227456 25925 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0857e43488c493c8d9860ae70fc04f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41531 } }
I20260807 20:16:57.227620 25925 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:57.227643 25925 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0857e43488c493c8d9860ae70fc04f2, State: Initialized, Role: FOLLOWER
I20260807 20:16:57.227715 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 25823
I20260807 20:16:57.227777 25925 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0857e43488c493c8d9860ae70fc04f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41531 } }
I20260807 20:16:57.227836 25925 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:16:57.227837 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:16:57.227857 25925 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:16:57.227907 25925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:57.228032 24355 external_mini_cluster.cc:1428] Setting key cfb0de3335df0f71fd8fd700977b2acf
I20260807 20:16:57.228511 25925 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0857e43488c493c8d9860ae70fc04f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41531 } }
I20260807 20:16:57.228653 25925 leader_election.cc:302] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0857e43488c493c8d9860ae70fc04f2; no voters:
I20260807 20:16:57.228820 25925 leader_election.cc:288] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:16:57.228997 25932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:57.229118 25925 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:16:57.229142 25932 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [term 1 LEADER]: Becoming Leader. State: Replica: f0857e43488c493c8d9860ae70fc04f2, State: Running, Role: LEADER
I20260807 20:16:57.229351 25932 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0857e43488c493c8d9860ae70fc04f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41531 } }
I20260807 20:16:57.229708 25932 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0857e43488c493c8d9860ae70fc04f2. Latest consensus state: current_term: 1 leader_uuid: "f0857e43488c493c8d9860ae70fc04f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0857e43488c493c8d9860ae70fc04f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41531 } } }
I20260807 20:16:57.229851 25932 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:57.230123 25944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:16:57.230822 25933 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0857e43488c493c8d9860ae70fc04f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0857e43488c493c8d9860ae70fc04f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41531 } } }
I20260807 20:16:57.230866 25944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:16:57.230921 25933 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2 [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:57.232733 25944 catalog_manager.cc:1347] Generated new cluster ID: 42d106bcb48e47e4a07c32dcc7ac2ea7
I20260807 20:16:57.232789 25944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:16:57 dist-test-slave-2j7r krb5kdc[25804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 20:16:57.238488 25930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:57.228663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:42725 (local address 127.0.0.1:41531)
0807 20:16:57.228869 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:57.228881 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:57.228920 (+ 39us) server_negotiation.cc:407] Connection header received
0807 20:16:57.229048 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:57.229056 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:57.229149 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:57.229302 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:57.230130 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:57.231528 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:57.232485 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:57.233019 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:57.233138 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:57.235350 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:57.235355 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:57.235355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:57.235366 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:57.236902 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:57.237316 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:57.237319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:57.237319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:57.237426 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:57.237632 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:57.237633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:57.237633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:57.237895 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:57.238057 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:57.238182 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 20:16:57.238283 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":116,"threads_started":1}
I20260807 20:16:57.239377 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:16:57.246619 25930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:57.240659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:51699 (local address 127.0.0.1:41531)
0807 20:16:57.240695 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:57.240697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:57.241246 (+ 549us) server_negotiation.cc:407] Connection header received
0807 20:16:57.241302 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:57.241303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:57.241314 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:57.241365 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:57.241847 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:57.242326 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:57.243002 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:57.243210 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:57.243265 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:57.244331 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:57.244332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:57.244332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:16:57.244339 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:57.245560 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:57.245892 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:57.245893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:57.245893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:57.245962 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:16:57.246150 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:16:57.246151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:16:57.246152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:16:57.246255 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:57.246360 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:57.246476 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 20:16:57.246533 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:16:57.255441 25944 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:16:57.255965 25944 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:16:57.263608 25944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0857e43488c493c8d9860ae70fc04f2: Generated new TSK 0
I20260807 20:16:57.263765 25944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:16:57.265658 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 25823
W20260807 20:16:57.283932 25802 connection.cc:535] client connection to 127.0.0.1:41531 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41531): Connection reset by peer (error 104)
2026-08-07T20:16:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:16:57 dist-test-slave-2j7r krb5kdc[25969](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:16:57 dist-test-slave-2j7r krb5kdc[25969](info): set up 2 sockets
Aug 07 20:16:57 dist-test-slave-2j7r krb5kdc[25969](info): commencing operation
krb5kdc: starting...
W20260807 20:16:58.418708 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:16:58 dist-test-slave-2j7r krb5kdc[25969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:16:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:16:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:16:58.543823 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46833
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:16:58.627497 25995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:16:58.627707 25995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:16:58.627740 25995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:16:58.628857 25995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:16:58.628897 25995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:16:58.628917 25995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:16:58.628932 25995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:16:58.628948 25995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:16:58.630435 25995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:16:58.630674 25995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:16:58.630883 25995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:16:58.635916 26001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:58.635941 25995 server_base.cc:1034] running on GCE node
W20260807 20:16:58.635995 26000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:16:58.636111 26004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:16:58.636399 25995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:16:58.636687 25995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:16:58.637854 25995 hybrid_clock.cc:648] HybridClock initialized: now 1786133818637837 us; error 27 us; skew 500 ppm
Aug 07 20:16:58 dist-test-slave-2j7r krb5kdc[25969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133818, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:16:58.640455 25995 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:16:58.640914 25995 webserver.cc:460] Webserver started at http://127.0.0.1:34807/ using document root <none> and password file <none>
I20260807 20:16:58.641153 25995 fs_manager.cc:359] Metadata directory not provided
I20260807 20:16:58.641206 25995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:16:58.641342 25995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:16:58.642407 25995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "9cc4b534034649dabdf788175afa936a"
format_stamp: "Formatted at 2026-08-07 20:16:58 on dist-test-slave-2j7r"
server_key: "d7a40f63d88ae98f2b4c07f882715398"
server_key_iv: "cf15d43b100927b9861113c4b59d91cb"
server_key_version: "encryptionkey@0"
I20260807 20:16:58.642740 25995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "9cc4b534034649dabdf788175afa936a"
format_stamp: "Formatted at 2026-08-07 20:16:58 on dist-test-slave-2j7r"
server_key: "d7a40f63d88ae98f2b4c07f882715398"
server_key_iv: "cf15d43b100927b9861113c4b59d91cb"
server_key_version: "encryptionkey@0"
I20260807 20:16:58.644037 25995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:16:58.644825 26018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:16:58.644975 25995 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:16:58.645066 25995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "9cc4b534034649dabdf788175afa936a"
format_stamp: "Formatted at 2026-08-07 20:16:58 on dist-test-slave-2j7r"
server_key: "d7a40f63d88ae98f2b4c07f882715398"
server_key_iv: "cf15d43b100927b9861113c4b59d91cb"
server_key_version: "encryptionkey@0"
I20260807 20:16:58.645159 25995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:16:58.667770 25995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:16:58.669991 25995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:16:58.670140 25995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:16:58.674149 25995 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46833
I20260807 20:16:58.674185 26089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46833 every 8 connection(s)
I20260807 20:16:58.674544 25995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:16:58.675442 26092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:16:58.677754 26092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a: Bootstrap starting.
I20260807 20:16:58.678303 26092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a: Neither blocks nor log segments found. Creating new log.
I20260807 20:16:58.678535 26092 log.cc:824] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a: Log is configured to *not* fsync() on all Append() calls
I20260807 20:16:58.679193 26092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a: No bootstrap required, opened a new log
I20260807 20:16:58.679387 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 25995
I20260807 20:16:58.679493 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:16:58.679674 24355 external_mini_cluster.cc:1428] Setting key fd8e2549f2a0c3a501662dd2a85b79b2
I20260807 20:16:58.681265 26092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc4b534034649dabdf788175afa936a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46833 } }
I20260807 20:16:58.681437 26092 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:16:58.681473 26092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cc4b534034649dabdf788175afa936a, State: Initialized, Role: FOLLOWER
I20260807 20:16:58.681631 26092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc4b534034649dabdf788175afa936a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46833 } }
I20260807 20:16:58.681710 26092 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:16:58.681731 26092 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:16:58.681777 26092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:16:58.682322 26092 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc4b534034649dabdf788175afa936a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46833 } }
I20260807 20:16:58.682410 26092 leader_election.cc:302] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cc4b534034649dabdf788175afa936a; no voters:
I20260807 20:16:58.682557 26092 leader_election.cc:288] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:16:58.682590 26100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:16:58.682745 26100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [term 1 LEADER]: Becoming Leader. State: Replica: 9cc4b534034649dabdf788175afa936a, State: Running, Role: LEADER
I20260807 20:16:58.682821 26092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:16:58.682842 26100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc4b534034649dabdf788175afa936a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46833 } }
I20260807 20:16:58.683189 26100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cc4b534034649dabdf788175afa936a. Latest consensus state: current_term: 1 leader_uuid: "9cc4b534034649dabdf788175afa936a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc4b534034649dabdf788175afa936a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46833 } } }
I20260807 20:16:58.683200 26103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cc4b534034649dabdf788175afa936a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc4b534034649dabdf788175afa936a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46833 } } }
I20260807 20:16:58.683372 26100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:58.683452 26103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a [sys.catalog]: This master's current role is: LEADER
I20260807 20:16:58.684046 26115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:16:58.684598 26115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:16:58.686285 26115 catalog_manager.cc:1347] Generated new cluster ID: 8d4470d4b03c42429eb7241fd990a15e
I20260807 20:16:58.686376 26115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:16:58.686548 26098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:58.680154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:34111 (local address 127.0.0.1:46833)
0807 20:16:58.680356 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:58.680368 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:58.680401 (+ 33us) server_negotiation.cc:407] Connection header received
0807 20:16:58.681223 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:58.681233 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:58.681301 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:58.681462 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:58.681683 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:58.683121 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:58.684203 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:58.684964 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:58.685067 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:58.685790 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:58.685795 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:58.685795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:58.686040 (+ 245us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:58.686078 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:58.686093 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:58.686182 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:58.686302 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 20:16:58.686331 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260807 20:16:58.687458 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:16:58.692168 26098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:16:58.688957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:50345 (local address 127.0.0.1:46833)
0807 20:16:58.689011 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 20:16:58.689012 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:16:58.689020 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:16:58.689039 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:16:58.689039 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:16:58.689051 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:16:58.689156 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:16:58.689285 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:58.689988 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:58.690935 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:16:58.691259 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:16:58.691333 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:16:58.691944 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:16:58.691945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:16:58.691946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:16:58.691950 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:16:58.691967 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:16:58.691972 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:16:58.692033 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:16:58.692086 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 20:16:58.692099 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:16:58.702868 26115 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:16:58.703614 26115 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:16:58.711738 26115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cc4b534034649dabdf788175afa936a: Generated new TSK 0
I20260807 20:16:58.711971 26115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:16:58.716744 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 25995
W20260807 20:16:58.733826 25967 connection.cc:535] client connection to 127.0.0.1:46833 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46833): Connection reset by peer (error 104)
2026-08-07T20:16:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1449 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:16:58 dist-test-slave-2j7r krb5kdc[26139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:16:58 dist-test-slave-2j7r krb5kdc[26139](info): set up 2 sockets
Aug 07 20:16:58 dist-test-slave-2j7r krb5kdc[26139](info): commencing operation
krb5kdc: starting...
W20260807 20:16:59.897605 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:16:59 dist-test-slave-2j7r krb5kdc[26139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:16:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:16:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:17:00.023685 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39839
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:17:00.107448 26161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:17:00.107661 26161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:17:00.107683 26161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:17:00.108803 26161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:17:00.108857 26161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:17:00.108884 26161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:17:00.108896 26161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:17:00.108907 26161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:17:00.110420 26161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:17:00.110625 26161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:17:00.110805 26161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:17:00.115859 26169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:00.116012 26161 server_base.cc:1034] running on GCE node
W20260807 20:17:00.115846 26172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:00.115830 26168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:00.116365 26161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:17:00.116683 26161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:17:00.117844 26161 hybrid_clock.cc:648] HybridClock initialized: now 1786133820117825 us; error 35 us; skew 500 ppm
Aug 07 20:17:00 dist-test-slave-2j7r krb5kdc[26139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133820, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:17:00.120597 26161 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:17:00.121122 26161 webserver.cc:460] Webserver started at http://127.0.0.1:43887/ using document root <none> and password file <none>
I20260807 20:17:00.121335 26161 fs_manager.cc:359] Metadata directory not provided
I20260807 20:17:00.121387 26161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:17:00.121513 26161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:17:00.122556 26161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "66270c6d5449400686fafd32f236ae63"
format_stamp: "Formatted at 2026-08-07 20:17:00 on dist-test-slave-2j7r"
server_key: "62b986374efed4685b0161fafa5c9326"
server_key_iv: "1090d1c537610dc0780dcc87fb551383"
server_key_version: "encryptionkey@0"
I20260807 20:17:00.122905 26161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "66270c6d5449400686fafd32f236ae63"
format_stamp: "Formatted at 2026-08-07 20:17:00 on dist-test-slave-2j7r"
server_key: "62b986374efed4685b0161fafa5c9326"
server_key_iv: "1090d1c537610dc0780dcc87fb551383"
server_key_version: "encryptionkey@0"
I20260807 20:17:00.124249 26161 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:17:00.125136 26181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:00.125314 26161 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:17:00.125412 26161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "66270c6d5449400686fafd32f236ae63"
format_stamp: "Formatted at 2026-08-07 20:17:00 on dist-test-slave-2j7r"
server_key: "62b986374efed4685b0161fafa5c9326"
server_key_iv: "1090d1c537610dc0780dcc87fb551383"
server_key_version: "encryptionkey@0"
I20260807 20:17:00.125490 26161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:17:00.128798 26161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:17:00.130970 26161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:17:00.131106 26161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:17:00.135147 26161 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39839
I20260807 20:17:00.135187 26255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39839 every 8 connection(s)
I20260807 20:17:00.135502 26161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:17:00.136526 26257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:17:00.138917 26257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63: Bootstrap starting.
I20260807 20:17:00.139170 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 26161
I20260807 20:17:00.139289 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:17:00.139482 24355 external_mini_cluster.cc:1428] Setting key 4893ac1d64d4fe42712b4bd0d076b90c
I20260807 20:17:00.139621 26257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63: Neither blocks nor log segments found. Creating new log.
I20260807 20:17:00.139935 26257 log.cc:824] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63: Log is configured to *not* fsync() on all Append() calls
I20260807 20:17:00.140705 26257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63: No bootstrap required, opened a new log
I20260807 20:17:00.142014 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:00.140000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:34537 (local address 127.0.0.1:39839)
0807 20:17:00.140207 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:00.140216 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:00.140251 (+ 35us) server_negotiation.cc:407] Connection header received
0807 20:17:00.140773 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:00.140782 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:00.140814 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:00.140956 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:00.141586 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:00.141590 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:00.141590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:17:00.141594 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:00.141631 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:17:00.141642 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:00.141709 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:00.141772 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 20:17:00.141809 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":77,"threads_started":1}
I20260807 20:17:00.142925 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:17:00.143047 26257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66270c6d5449400686fafd32f236ae63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39839 } }
I20260807 20:17:00.143208 26257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:17:00.143244 26257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66270c6d5449400686fafd32f236ae63, State: Initialized, Role: FOLLOWER
I20260807 20:17:00.143369 26257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66270c6d5449400686fafd32f236ae63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39839 } }
I20260807 20:17:00.143456 26257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:17:00.143493 26257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:17:00.143537 26257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:17:00.144274 26257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66270c6d5449400686fafd32f236ae63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39839 } }
I20260807 20:17:00.144416 26257 leader_election.cc:302] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66270c6d5449400686fafd32f236ae63; no voters:
I20260807 20:17:00.144583 26257 leader_election.cc:288] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:17:00.144615 26274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:17:00.144855 26274 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [term 1 LEADER]: Becoming Leader. State: Replica: 66270c6d5449400686fafd32f236ae63, State: Running, Role: LEADER
I20260807 20:17:00.144932 26257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:17:00.145030 26274 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66270c6d5449400686fafd32f236ae63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39839 } }
I20260807 20:17:00.145394 26274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66270c6d5449400686fafd32f236ae63. Latest consensus state: current_term: 1 leader_uuid: "66270c6d5449400686fafd32f236ae63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66270c6d5449400686fafd32f236ae63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39839 } } }
I20260807 20:17:00.145627 26274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:00.145643 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:00.144444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:51839 (local address 127.0.0.1:39839)
0807 20:17:00.144491 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:00.144492 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:00.144621 (+ 129us) server_negotiation.cc:407] Connection header received
0807 20:17:00.144680 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:00.144681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:00.144686 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:00.144729 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:00.145412 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:00.145414 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:00.145414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:17:00.145419 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:00.145434 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:17:00.145439 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:00.145467 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:00.145570 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 20:17:00.145584 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:17:00.145551 26277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66270c6d5449400686fafd32f236ae63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66270c6d5449400686fafd32f236ae63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39839 } } }
I20260807 20:17:00.146255 26277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63 [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:00.146394 26287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:17:00.146864 26287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:17:00.148489 26287 catalog_manager.cc:1347] Generated new cluster ID: f2c65edfae37491995168c4e10c41daa
I20260807 20:17:00.148545 26287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:17:00.168581 26287 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:17:00.168779 26287 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:17:00.171741 26287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66270c6d5449400686fafd32f236ae63: Generated new TSK 0
I20260807 20:17:00.171954 26287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:17:00.180423 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 26161
2026-08-07T20:17:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1470 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:17:00 dist-test-slave-2j7r krb5kdc[26302](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:17:00 dist-test-slave-2j7r krb5kdc[26302](info): set up 2 sockets
Aug 07 20:17:00 dist-test-slave-2j7r krb5kdc[26302](info): commencing operation
krb5kdc: starting...
W20260807 20:17:01.354173 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:17:01 dist-test-slave-2j7r krb5kdc[26302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:17:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:17:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:17:01.484790 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33541
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:17:01.569410 26326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:17:01.569622 26326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:17:01.569643 26326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:17:01.570869 26326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:17:01.570916 26326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:17:01.570930 26326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:17:01.570941 26326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:17:01.570952 26326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:17:01.572431 26326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33541
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:17:01.572669 26326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:17:01.572890 26326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:17:01.578030 26334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:01.578030 26339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:01.578053 26335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:01.578095 26326 server_base.cc:1034] running on GCE node
I20260807 20:17:01.578500 26326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:17:01.578773 26326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:17:01.579957 26326 hybrid_clock.cc:648] HybridClock initialized: now 1786133821579942 us; error 30 us; skew 500 ppm
Aug 07 20:17:01 dist-test-slave-2j7r krb5kdc[26302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133821, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:17:01.582489 26326 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:17:01.582953 26326 webserver.cc:460] Webserver started at http://127.0.0.1:44953/ using document root <none> and password file <none>
I20260807 20:17:01.583168 26326 fs_manager.cc:359] Metadata directory not provided
I20260807 20:17:01.583215 26326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:17:01.583340 26326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:17:01.584403 26326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "89bdb010f24b4021a2acb3aa2c40a97a"
format_stamp: "Formatted at 2026-08-07 20:17:01 on dist-test-slave-2j7r"
server_key: "2939abd34c6bf42c82d199288a0d1e36"
server_key_iv: "d7bcca6eaa9bb6edbefc1937e5ddaa94"
server_key_version: "encryptionkey@0"
I20260807 20:17:01.584728 26326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "89bdb010f24b4021a2acb3aa2c40a97a"
format_stamp: "Formatted at 2026-08-07 20:17:01 on dist-test-slave-2j7r"
server_key: "2939abd34c6bf42c82d199288a0d1e36"
server_key_iv: "d7bcca6eaa9bb6edbefc1937e5ddaa94"
server_key_version: "encryptionkey@0"
I20260807 20:17:01.586076 26326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:17:01.586758 26350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:01.586915 26326 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:17:01.587005 26326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "89bdb010f24b4021a2acb3aa2c40a97a"
format_stamp: "Formatted at 2026-08-07 20:17:01 on dist-test-slave-2j7r"
server_key: "2939abd34c6bf42c82d199288a0d1e36"
server_key_iv: "d7bcca6eaa9bb6edbefc1937e5ddaa94"
server_key_version: "encryptionkey@0"
I20260807 20:17:01.587064 26326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:17:01.607609 26326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:17:01.609733 26326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:17:01.609905 26326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:17:01.614076 26326 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33541
I20260807 20:17:01.614120 26435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33541 every 8 connection(s)
I20260807 20:17:01.614516 26326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:17:01.615423 26436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:17:01.617794 26436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a: Bootstrap starting.
I20260807 20:17:01.618338 26436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a: Neither blocks nor log segments found. Creating new log.
I20260807 20:17:01.618569 26436 log.cc:824] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a: Log is configured to *not* fsync() on all Append() calls
I20260807 20:17:01.619259 26436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a: No bootstrap required, opened a new log
I20260807 20:17:01.620820 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 26326
I20260807 20:17:01.620957 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:17:01.620947 26436 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bdb010f24b4021a2acb3aa2c40a97a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33541 } }
I20260807 20:17:01.621179 24355 external_mini_cluster.cc:1428] Setting key 031381f96641de06a8fbb302a027341c
I20260807 20:17:01.621173 26436 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:17:01.621212 26436 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bdb010f24b4021a2acb3aa2c40a97a, State: Initialized, Role: FOLLOWER
I20260807 20:17:01.621363 26436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bdb010f24b4021a2acb3aa2c40a97a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33541 } }
I20260807 20:17:01.621438 26436 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:17:01.621467 26436 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:17:01.621515 26436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:17:01.622388 26436 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bdb010f24b4021a2acb3aa2c40a97a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33541 } }
I20260807 20:17:01.622483 26436 leader_election.cc:302] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89bdb010f24b4021a2acb3aa2c40a97a; no voters:
I20260807 20:17:01.622640 26436 leader_election.cc:288] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:17:01.622687 26444 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:17:01.622848 26444 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [term 1 LEADER]: Becoming Leader. State: Replica: 89bdb010f24b4021a2acb3aa2c40a97a, State: Running, Role: LEADER
I20260807 20:17:01.622907 26436 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:17:01.623039 26444 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bdb010f24b4021a2acb3aa2c40a97a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33541 } }
I20260807 20:17:01.624245 26447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89bdb010f24b4021a2acb3aa2c40a97a. Latest consensus state: current_term: 1 leader_uuid: "89bdb010f24b4021a2acb3aa2c40a97a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bdb010f24b4021a2acb3aa2c40a97a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33541 } } }
I20260807 20:17:01.624459 26447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:01.624642 26448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89bdb010f24b4021a2acb3aa2c40a97a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89bdb010f24b4021a2acb3aa2c40a97a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33541 } } }
I20260807 20:17:01.624752 26448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a [sys.catalog]: This master's current role is: LEADER
W20260807 20:17:01.624781 26467 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:17:01.624836 26467 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:17:01.624913 26468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:17:01.625077 26468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:17:01.627135 26468 catalog_manager.cc:1347] Generated new cluster ID: 927a3c4d943a4a108d59ff618e1e9b6e
I20260807 20:17:01.627189 26468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:17:01 dist-test-slave-2j7r krb5kdc[26302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 20:17:01.632012 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:01.621648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:40395 (local address 127.0.0.1:33541)
0807 20:17:01.621856 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:01.621866 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:01.621910 (+ 44us) server_negotiation.cc:407] Connection header received
0807 20:17:01.622031 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:01.622037 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:01.622083 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:01.622199 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:01.623018 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:01.624474 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:01.625451 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:01.626081 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:01.626192 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:01.628686 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:01.628691 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:01.628691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:01.628704 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:01.630418 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:01.630787 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:01.630790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:01.630790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:01.630912 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:01.631122 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:01.631123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:01.631123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:01.631362 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:01.631515 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:01.631647 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 20:17:01.631853 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260807 20:17:01.632925 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:17:01.634554 26468 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:17:01.635097 26468 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:17:01.641347 26468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89bdb010f24b4021a2acb3aa2c40a97a: Generated new TSK 0
I20260807 20:17:01.641518 26468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:17:01.644239 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:01.637406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:41487 (local address 127.0.0.1:33541)
0807 20:17:01.637442 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:01.637443 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:01.637556 (+ 113us) server_negotiation.cc:407] Connection header received
0807 20:17:01.637619 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:01.637620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:01.637629 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:01.637689 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:01.638322 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:01.639081 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:01.640103 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:01.640393 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:01.640447 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:01.641844 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:01.641845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:01.641846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:01.641852 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:01.643173 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:01.643566 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:01.643567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:01.643567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:01.643611 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:01.643795 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:01.643796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:01.643796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:01.643901 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:01.643973 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:01.644109 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 20:17:01.644156 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:17:01.646104 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 26326
W20260807 20:17:01.663977 26300 connection.cc:535] client connection to 127.0.0.1:33541 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33541): Connection reset by peer (error 104)
2026-08-07T20:17:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:17:01 dist-test-slave-2j7r krb5kdc[26484](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:17:01 dist-test-slave-2j7r krb5kdc[26484](info): set up 2 sockets
Aug 07 20:17:01 dist-test-slave-2j7r krb5kdc[26484](info): commencing operation
krb5kdc: starting...
W20260807 20:17:02.817118 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:17:02 dist-test-slave-2j7r krb5kdc[26484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:17:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:17:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:17:02.940693 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:17:03.023248 26503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:17:03.023464 26503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:17:03.023495 26503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:17:03.024632 26503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:17:03.024675 26503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:17:03.024693 26503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:17:03.024709 26503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:17:03.024724 26503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:17:03.026309 26503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44195
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:17:03.026576 26503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:17:03.026784 26503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:17:03.031642 26512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:03.031674 26510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:03.031672 26516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:03.031730 26503 server_base.cc:1034] running on GCE node
I20260807 20:17:03.032104 26503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:17:03.032395 26503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:17:03.033545 26503 hybrid_clock.cc:648] HybridClock initialized: now 1786133823033522 us; error 37 us; skew 500 ppm
Aug 07 20:17:03 dist-test-slave-2j7r krb5kdc[26484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:17:03.035995 26503 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:17:03.036434 26503 webserver.cc:460] Webserver started at http://127.0.0.1:41185/ using document root <none> and password file <none>
I20260807 20:17:03.036644 26503 fs_manager.cc:359] Metadata directory not provided
I20260807 20:17:03.036682 26503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:17:03.036779 26503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:17:03.037791 26503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "1cbd57368bef461dabe8d8c5d9e2afc1"
format_stamp: "Formatted at 2026-08-07 20:17:03 on dist-test-slave-2j7r"
server_key: "3be571c39da08aba1269fa27cb055199"
server_key_iv: "08d709850f0835575dfd96ab5f6f87ac"
server_key_version: "encryptionkey@0"
I20260807 20:17:03.038075 26503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "1cbd57368bef461dabe8d8c5d9e2afc1"
format_stamp: "Formatted at 2026-08-07 20:17:03 on dist-test-slave-2j7r"
server_key: "3be571c39da08aba1269fa27cb055199"
server_key_iv: "08d709850f0835575dfd96ab5f6f87ac"
server_key_version: "encryptionkey@0"
I20260807 20:17:03.039232 26503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:17:03.039921 26528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:03.040064 26503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:17:03.040150 26503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "1cbd57368bef461dabe8d8c5d9e2afc1"
format_stamp: "Formatted at 2026-08-07 20:17:03 on dist-test-slave-2j7r"
server_key: "3be571c39da08aba1269fa27cb055199"
server_key_iv: "08d709850f0835575dfd96ab5f6f87ac"
server_key_version: "encryptionkey@0"
I20260807 20:17:03.040210 26503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:17:03.070693 26503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:17:03.072532 26503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:17:03.072681 26503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:17:03.076711 26503 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44195
I20260807 20:17:03.076749 26607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44195 every 8 connection(s)
I20260807 20:17:03.077119 26503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:17:03.078019 26609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:17:03.080252 26609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1: Bootstrap starting.
I20260807 20:17:03.080824 26609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1: Neither blocks nor log segments found. Creating new log.
I20260807 20:17:03.081108 26609 log.cc:824] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1: Log is configured to *not* fsync() on all Append() calls
I20260807 20:17:03.081771 26609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1: No bootstrap required, opened a new log
I20260807 20:17:03.083294 26609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbd57368bef461dabe8d8c5d9e2afc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } }
I20260807 20:17:03.083458 26609 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:17:03.083492 26609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cbd57368bef461dabe8d8c5d9e2afc1, State: Initialized, Role: FOLLOWER
I20260807 20:17:03.083633 26609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbd57368bef461dabe8d8c5d9e2afc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } }
I20260807 20:17:03.083712 26609 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:17:03.083746 26609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:17:03.083792 26609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:17:03.084275 26609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbd57368bef461dabe8d8c5d9e2afc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } }
I20260807 20:17:03.084363 26609 leader_election.cc:302] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cbd57368bef461dabe8d8c5d9e2afc1; no voters:
I20260807 20:17:03.084522 26609 leader_election.cc:288] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:17:03.084553 26616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:17:03.084702 26616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [term 1 LEADER]: Becoming Leader. State: Replica: 1cbd57368bef461dabe8d8c5d9e2afc1, State: Running, Role: LEADER
I20260807 20:17:03.084774 26609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:17:03.084811 26616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbd57368bef461dabe8d8c5d9e2afc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } }
I20260807 20:17:03.085410 26617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cbd57368bef461dabe8d8c5d9e2afc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbd57368bef461dabe8d8c5d9e2afc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } } }
I20260807 20:17:03.085497 26617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:03.085585 26620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cbd57368bef461dabe8d8c5d9e2afc1. Latest consensus state: current_term: 1 leader_uuid: "1cbd57368bef461dabe8d8c5d9e2afc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbd57368bef461dabe8d8c5d9e2afc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44195 } } }
I20260807 20:17:03.085739 26620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1 [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:03.085908 26631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:17:03.086285 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 26503
I20260807 20:17:03.086404 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:17:03.086429 26631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:17:03.086584 24355 external_mini_cluster.cc:1428] Setting key 11cf5be9b78aa0903843d00de12f7bb3
I20260807 20:17:03.087970 26631 catalog_manager.cc:1347] Generated new cluster ID: c96b2ef6d1b14f55bab14f329e246960
I20260807 20:17:03.088020 26631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:17:03.093813 26645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:03.087018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:50139 (local address 127.0.0.1:44195)
0807 20:17:03.087390 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:03.087398 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:03.087440 (+ 42us) server_negotiation.cc:407] Connection header received
0807 20:17:03.087542 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:03.087548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:03.087603 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:03.087747 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:03.089184 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:03.090639 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:03.091683 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:03.092319 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:03.092433 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:03.093076 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:03.093080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:03.093081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:17:03.093339 (+ 258us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:03.093380 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:17:03.093395 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:03.093481 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:03.093555 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 20:17:03.093584 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":76,"threads_started":1}
I20260807 20:17:03.094755 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:17:03.099648 26645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:03.096373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:60333 (local address 127.0.0.1:44195)
0807 20:17:03.096435 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:03.096436 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:03.096444 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:17:03.096536 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:03.096537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:03.096547 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:03.096596 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:03.096725 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:03.097463 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:03.098466 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:03.098768 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:03.098816 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:03.099423 (+ 607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:03.099424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:03.099424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:17:03.099428 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:03.099441 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:17:03.099446 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:03.099493 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:03.099566 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 20:17:03.099578 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 20:17:03.109328 26631 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:17:03.109997 26631 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:17:03.119037 26631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cbd57368bef461dabe8d8c5d9e2afc1: Generated new TSK 0
I20260807 20:17:03.119221 26631 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:17:03.125011 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 26503
W20260807 20:17:03.136267 26480 connection.cc:535] client connection to 127.0.0.1:44195 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44195): Connection reset by peer (error 104)
2026-08-07T20:17:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:17:03 dist-test-slave-2j7r krb5kdc[26662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:17:03 dist-test-slave-2j7r krb5kdc[26662](info): set up 2 sockets
Aug 07 20:17:03 dist-test-slave-2j7r krb5kdc[26662](info): commencing operation
krb5kdc: starting...
W20260807 20:17:04.289835 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:17:04 dist-test-slave-2j7r krb5kdc[26662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:17:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:17:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:17:04.415684 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33733
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:17:04.498674 26680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:17:04.498908 26680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:17:04.498932 26680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:17:04.500069 26680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:17:04.500111 26680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:17:04.500124 26680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:17:04.500136 26680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:17:04.500147 26680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:17:04.501680 26680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33733
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:17:04.501925 26680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:17:04.502135 26680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:17:04.506997 26687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:04.507004 26686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:04.507010 26691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:04.507093 26680 server_base.cc:1034] running on GCE node
I20260807 20:17:04.507440 26680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:17:04.507709 26680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:17:04.508881 26680 hybrid_clock.cc:648] HybridClock initialized: now 1786133824508862 us; error 37 us; skew 500 ppm
Aug 07 20:17:04 dist-test-slave-2j7r krb5kdc[26662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133824, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:17:04.511746 26680 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:17:04.512164 26680 webserver.cc:460] Webserver started at http://127.0.0.1:37909/ using document root <none> and password file <none>
I20260807 20:17:04.512373 26680 fs_manager.cc:359] Metadata directory not provided
I20260807 20:17:04.512420 26680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:17:04.512521 26680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:17:04.513633 26680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "d65915aba8ad4503a09f5dcc26258c4f"
format_stamp: "Formatted at 2026-08-07 20:17:04 on dist-test-slave-2j7r"
server_key: "af09ceef25d94d630a6f523d8f165f71"
server_key_iv: "5603d7ff9e5c45a52f5c7fa3bad67c5e"
server_key_version: "encryptionkey@0"
I20260807 20:17:04.513952 26680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "d65915aba8ad4503a09f5dcc26258c4f"
format_stamp: "Formatted at 2026-08-07 20:17:04 on dist-test-slave-2j7r"
server_key: "af09ceef25d94d630a6f523d8f165f71"
server_key_iv: "5603d7ff9e5c45a52f5c7fa3bad67c5e"
server_key_version: "encryptionkey@0"
I20260807 20:17:04.515307 26680 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:17:04.516040 26703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:04.516237 26680 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:17:04.516371 26680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "d65915aba8ad4503a09f5dcc26258c4f"
format_stamp: "Formatted at 2026-08-07 20:17:04 on dist-test-slave-2j7r"
server_key: "af09ceef25d94d630a6f523d8f165f71"
server_key_iv: "5603d7ff9e5c45a52f5c7fa3bad67c5e"
server_key_version: "encryptionkey@0"
I20260807 20:17:04.516433 26680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:17:04.519651 26680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:17:04.521602 26680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:17:04.521708 26680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:17:04.525424 26680 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33733
I20260807 20:17:04.525457 26781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33733 every 8 connection(s)
I20260807 20:17:04.525748 26680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:17:04.526734 26783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:17:04.528990 26783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f: Bootstrap starting.
I20260807 20:17:04.529560 26783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f: Neither blocks nor log segments found. Creating new log.
I20260807 20:17:04.529819 26783 log.cc:824] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f: Log is configured to *not* fsync() on all Append() calls
I20260807 20:17:04.530485 26783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f: No bootstrap required, opened a new log
I20260807 20:17:04.531131 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 26680
I20260807 20:17:04.531262 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:17:04.531462 24355 external_mini_cluster.cc:1428] Setting key 8523e4c50ff3674920457817a53c755b
I20260807 20:17:04.532426 26783 raft_consensus.cc:348] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65915aba8ad4503a09f5dcc26258c4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } }
I20260807 20:17:04.532557 26783 raft_consensus.cc:374] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:17:04.532588 26783 raft_consensus.cc:729] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d65915aba8ad4503a09f5dcc26258c4f, State: Initialized, Role: FOLLOWER
I20260807 20:17:04.532729 26783 consensus_queue.cc:260] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65915aba8ad4503a09f5dcc26258c4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } }
I20260807 20:17:04.532835 26783 raft_consensus.cc:388] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:17:04.532876 26783 raft_consensus.cc:482] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:17:04.532922 26783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:17:04.533654 26789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:04.531972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:50355 (local address 127.0.0.1:33733)
0807 20:17:04.532167 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:04.532176 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:04.532200 (+ 24us) server_negotiation.cc:407] Connection header received
0807 20:17:04.532399 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:04.532406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:04.532432 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:04.532552 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:04.533217 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:04.533221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:04.533222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:17:04.533226 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:04.533269 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:17:04.533280 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:04.533345 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:04.533429 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 20:17:04.533471 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260807 20:17:04.533825 26783 raft_consensus.cc:504] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65915aba8ad4503a09f5dcc26258c4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } }
I20260807 20:17:04.533932 26783 leader_election.cc:302] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d65915aba8ad4503a09f5dcc26258c4f; no voters:
I20260807 20:17:04.534086 26783 leader_election.cc:288] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:17:04.534111 26792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:17:04.534281 26792 raft_consensus.cc:686] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [term 1 LEADER]: Becoming Leader. State: Replica: d65915aba8ad4503a09f5dcc26258c4f, State: Running, Role: LEADER
I20260807 20:17:04.534394 26783 sys_catalog.cc:564] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:17:04.534524 26792 consensus_queue.cc:237] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65915aba8ad4503a09f5dcc26258c4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } }
I20260807 20:17:04.534559 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:17:04.534904 26794 sys_catalog.cc:455] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d65915aba8ad4503a09f5dcc26258c4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65915aba8ad4503a09f5dcc26258c4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } } }
I20260807 20:17:04.535099 26794 sys_catalog.cc:458] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:04.535043 26796 sys_catalog.cc:455] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d65915aba8ad4503a09f5dcc26258c4f. Latest consensus state: current_term: 1 leader_uuid: "d65915aba8ad4503a09f5dcc26258c4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65915aba8ad4503a09f5dcc26258c4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } } }
I20260807 20:17:04.535511 26796 sys_catalog.cc:458] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:04.535812 26809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:17:04.536379 26809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:17:04.537016 26789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:04.536048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:57903 (local address 127.0.0.1:33733)
0807 20:17:04.536076 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:04.536078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:04.536225 (+ 147us) server_negotiation.cc:407] Connection header received
0807 20:17:04.536277 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:04.536278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:04.536284 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:04.536329 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:04.536830 (+ 501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:04.536832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:04.536832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:17:04.536836 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:04.536848 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:17:04.536852 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:04.536899 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:04.536945 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 20:17:04.536958 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260807 20:17:04.537818 26809 catalog_manager.cc:1347] Generated new cluster ID: a8c29e0c0c1c4773874e78cfa028c9ec
I20260807 20:17:04.537868 26809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:17:04.542973 26809 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:17:04.543121 26809 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:17:04.545048 26809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d65915aba8ad4503a09f5dcc26258c4f: Generated new TSK 0
I20260807 20:17:04.545276 26809 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:17:04.552035 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 26680
2026-08-07T20:17:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1429 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:17:04 dist-test-slave-2j7r krb5kdc[26830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:17:04 dist-test-slave-2j7r krb5kdc[26830](info): set up 2 sockets
Aug 07 20:17:04 dist-test-slave-2j7r krb5kdc[26830](info): commencing operation
krb5kdc: starting...
W20260807 20:17:05.714282 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:17:05 dist-test-slave-2j7r krb5kdc[26830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:17:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:17:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:17:05.840255 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:46585
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:34849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:46585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:17:05.921139 26850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:17:05.921347 26850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:17:05.921378 26850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:17:05.922502 26850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:17:05.922545 26850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:17:05.922564 26850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:17:05.922580 26850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:17:05.922595 26850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:17:05.924060 26850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:34849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:46585
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:46585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:17:05.924312 26850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:17:05.924553 26850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:17:05.929713 26850 server_base.cc:1034] running on GCE node
W20260807 20:17:05.929628 26861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:05.929597 26864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:05.929620 26860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:05.930073 26850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:17:05.930370 26850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:17:05.931529 26850 hybrid_clock.cc:648] HybridClock initialized: now 1786133825931508 us; error 39 us; skew 500 ppm
Aug 07 20:17:05 dist-test-slave-2j7r krb5kdc[26830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133825, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:17:05.937695 26850 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:17:05.938154 26850 webserver.cc:460] Webserver started at http://127.23.200.254:37015/ using document root <none> and password file <none>
I20260807 20:17:05.938338 26850 fs_manager.cc:359] Metadata directory not provided
I20260807 20:17:05.938377 26850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:17:05.938473 26850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:17:05.939560 26850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "beb1d04a0e914155b7e9d646398e8a0c"
format_stamp: "Formatted at 2026-08-07 20:17:05 on dist-test-slave-2j7r"
server_key: "50e663a5bac346b697602890cffb0adc"
server_key_iv: "1932523594d03ddd8d15139c24294a6c"
server_key_version: "encryptionkey@0"
I20260807 20:17:05.939875 26850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "beb1d04a0e914155b7e9d646398e8a0c"
format_stamp: "Formatted at 2026-08-07 20:17:05 on dist-test-slave-2j7r"
server_key: "50e663a5bac346b697602890cffb0adc"
server_key_iv: "1932523594d03ddd8d15139c24294a6c"
server_key_version: "encryptionkey@0"
I20260807 20:17:05.941118 26850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:17:05.941826 26873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:05.941998 26850 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:17:05.942085 26850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "beb1d04a0e914155b7e9d646398e8a0c"
format_stamp: "Formatted at 2026-08-07 20:17:05 on dist-test-slave-2j7r"
server_key: "50e663a5bac346b697602890cffb0adc"
server_key_iv: "1932523594d03ddd8d15139c24294a6c"
server_key_version: "encryptionkey@0"
I20260807 20:17:05.942142 26850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:17:05.969339 26850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:17:05.971256 26850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:17:05.971380 26850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:17:05.975359 26850 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:46585
I20260807 20:17:05.975448 26952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:46585 every 8 connection(s)
I20260807 20:17:05.975772 26850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:17:05.976668 26953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:17:05.979094 26953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c: Bootstrap starting.
I20260807 20:17:05.979676 26953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c: Neither blocks nor log segments found. Creating new log.
I20260807 20:17:05.979939 26953 log.cc:824] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c: Log is configured to *not* fsync() on all Append() calls
I20260807 20:17:05.980629 26953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c: No bootstrap required, opened a new log
I20260807 20:17:05.982252 26953 raft_consensus.cc:348] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb1d04a0e914155b7e9d646398e8a0c" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 46585 } }
I20260807 20:17:05.982395 26953 raft_consensus.cc:374] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:17:05.982417 26953 raft_consensus.cc:729] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb1d04a0e914155b7e9d646398e8a0c, State: Initialized, Role: FOLLOWER
I20260807 20:17:05.982564 26953 consensus_queue.cc:260] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb1d04a0e914155b7e9d646398e8a0c" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 46585 } }
I20260807 20:17:05.982648 26953 raft_consensus.cc:388] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:17:05.982672 26953 raft_consensus.cc:482] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:17:05.982704 26953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:17:05.983539 26953 raft_consensus.cc:504] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb1d04a0e914155b7e9d646398e8a0c" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 46585 } }
I20260807 20:17:05.983630 26953 leader_election.cc:302] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beb1d04a0e914155b7e9d646398e8a0c; no voters:
I20260807 20:17:05.983778 26953 leader_election.cc:288] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:17:05.983870 26956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:17:05.984028 26956 raft_consensus.cc:686] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [term 1 LEADER]: Becoming Leader. State: Replica: beb1d04a0e914155b7e9d646398e8a0c, State: Running, Role: LEADER
I20260807 20:17:05.984071 26953 sys_catalog.cc:564] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:17:05.984118 26956 consensus_queue.cc:237] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb1d04a0e914155b7e9d646398e8a0c" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 46585 } }
I20260807 20:17:05.984517 26957 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "beb1d04a0e914155b7e9d646398e8a0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb1d04a0e914155b7e9d646398e8a0c" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 46585 } } }
I20260807 20:17:05.984611 26957 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:05.984794 26958 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader beb1d04a0e914155b7e9d646398e8a0c. Latest consensus state: current_term: 1 leader_uuid: "beb1d04a0e914155b7e9d646398e8a0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb1d04a0e914155b7e9d646398e8a0c" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 46585 } } }
I20260807 20:17:05.984885 26958 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:05.984839 26976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:17:05.985496 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 26850
I20260807 20:17:05.985531 26976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:17:05.985630 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:17:05.985817 24355 external_mini_cluster.cc:1428] Setting key 7acc498f90e96c9cbd4a02bae5d120f6
I20260807 20:17:05.987133 26976 catalog_manager.cc:1347] Generated new cluster ID: f4ab0929927445f0a10828d8722c39c6
I20260807 20:17:05.987186 26976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:17:05 dist-test-slave-2j7r krb5kdc[26830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:17:05.997629 26992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:05.986287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60267 (local address 127.23.200.254:46585)
0807 20:17:05.986461 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:05.986470 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:05.986607 (+ 137us) server_negotiation.cc:407] Connection header received
0807 20:17:05.986751 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:05.986758 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:05.986804 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:05.986930 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:05.987706 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:05.989233 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:05.990277 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:05.990963 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:05.991081 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:05.993976 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:05.993981 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:05.993982 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:05.993995 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:05.995891 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:05.996346 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:05.996350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:05.996350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:05.996461 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:05.996673 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:05.996674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:05.996674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:05.996967 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:05.997140 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:05.997275 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 20:17:05.997378 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260807 20:17:05.998907 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:17:05.999431 26976 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:17:06.000118 26976 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:17:06.006167 26976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P beb1d04a0e914155b7e9d646398e8a0c: Generated new TSK 0
I20260807 20:17:06.006351 26976 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:17:06.008026 26992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:06.000361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58399 (local address 127.23.200.254:46585)
0807 20:17:06.000401 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:06.000403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:06.000537 (+ 134us) server_negotiation.cc:407] Connection header received
0807 20:17:06.000581 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:06.000582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:06.000591 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:06.000645 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:06.001276 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:06.002083 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:06.003126 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:06.003442 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:06.003491 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:06.004827 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:06.004828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:06.004829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:06.004835 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:06.006896 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:06.007299 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:06.007299 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:06.007300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:06.007346 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:06.007542 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:06.007542 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:06.007542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:06.007629 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:06.007699 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:06.007845 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 20:17:06.007914 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 20:17:06.009958 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 26850
W20260807 20:17:06.022459 26827 connection.cc:535] client connection to 127.23.200.254:46585 recv error: Network error: failed to read from TLS socket (remote: 127.23.200.254:46585): Connection reset by peer (error 104)
2026-08-07T20:17:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:17:06 dist-test-slave-2j7r krb5kdc[27006](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:17:06 dist-test-slave-2j7r krb5kdc[27006](info): set up 2 sockets
Aug 07 20:17:06 dist-test-slave-2j7r krb5kdc[27006](info): commencing operation
krb5kdc: starting...
W20260807 20:17:07.168934 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:17:07 dist-test-slave-2j7r krb5kdc[27006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:17:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:17:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:17:07.295712 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:42029
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:42029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:17:07.379869 27028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:17:07.380076 27028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:17:07.380097 27028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:17:07.381276 27028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:17:07.381317 27028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:17:07.381337 27028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:17:07.381349 27028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:17:07.381363 27028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:17:07.382839 27028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:42029
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:42029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:17:07.383068 27028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:17:07.383280 27028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:17:07.388083 27033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:07.388130 27036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:07.388108 27042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:07.388144 27028 server_base.cc:1034] running on GCE node
I20260807 20:17:07.388566 27028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:17:07.388832 27028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:17:07.389991 27028 hybrid_clock.cc:648] HybridClock initialized: now 1786133827389975 us; error 30 us; skew 500 ppm
Aug 07 20:17:07 dist-test-slave-2j7r krb5kdc[27006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133827, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:17:07.392418 27028 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:17:07.392838 27028 webserver.cc:460] Webserver started at http://127.23.200.254:33415/ using document root <none> and password file <none>
I20260807 20:17:07.393035 27028 fs_manager.cc:359] Metadata directory not provided
I20260807 20:17:07.393100 27028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:17:07.393229 27028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:17:07.394243 27028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "e0ca2a5b38ce4829b259a685ee93c3a5"
format_stamp: "Formatted at 2026-08-07 20:17:07 on dist-test-slave-2j7r"
server_key: "cb3048daac4b15fd14170be479f4b3f0"
server_key_iv: "87881739c1ced11e1cb908d9aa4b781c"
server_key_version: "encryptionkey@0"
I20260807 20:17:07.394590 27028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "e0ca2a5b38ce4829b259a685ee93c3a5"
format_stamp: "Formatted at 2026-08-07 20:17:07 on dist-test-slave-2j7r"
server_key: "cb3048daac4b15fd14170be479f4b3f0"
server_key_iv: "87881739c1ced11e1cb908d9aa4b781c"
server_key_version: "encryptionkey@0"
I20260807 20:17:07.395915 27028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 20:17:07.396610 27052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:07.396768 27028 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:17:07.396880 27028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "e0ca2a5b38ce4829b259a685ee93c3a5"
format_stamp: "Formatted at 2026-08-07 20:17:07 on dist-test-slave-2j7r"
server_key: "cb3048daac4b15fd14170be479f4b3f0"
server_key_iv: "87881739c1ced11e1cb908d9aa4b781c"
server_key_version: "encryptionkey@0"
I20260807 20:17:07.396943 27028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:17:07.412174 27028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:17:07.414127 27028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:17:07.414268 27028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:17:07.418151 27028 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:42029
I20260807 20:17:07.418180 27133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:42029 every 8 connection(s)
I20260807 20:17:07.418524 27028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:17:07.419483 27134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:17:07.421188 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 27028
I20260807 20:17:07.421320 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:17:07.421516 24355 external_mini_cluster.cc:1428] Setting key e11a62f086613fd73e3d21ce53de99da
I20260807 20:17:07.422140 27134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5: Bootstrap starting.
I20260807 20:17:07.422730 27134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5: Neither blocks nor log segments found. Creating new log.
I20260807 20:17:07.423040 27134 log.cc:824] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5: Log is configured to *not* fsync() on all Append() calls
I20260807 20:17:07.423745 27134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5: No bootstrap required, opened a new log
I20260807 20:17:07.425926 27134 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ca2a5b38ce4829b259a685ee93c3a5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42029 } }
I20260807 20:17:07.426106 27134 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:17:07.426153 27134 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0ca2a5b38ce4829b259a685ee93c3a5, State: Initialized, Role: FOLLOWER
I20260807 20:17:07.426378 27134 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ca2a5b38ce4829b259a685ee93c3a5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42029 } }
I20260807 20:17:07.426487 27134 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:17:07.426528 27134 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:17:07.426573 27134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:17:07.427141 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:07.422065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54493 (local address 127.23.200.254:42029)
0807 20:17:07.422243 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:07.422253 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:07.422285 (+ 32us) server_negotiation.cc:407] Connection header received
0807 20:17:07.422714 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:07.422720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:07.422766 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:07.422907 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:07.423166 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:07.424141 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:07.425235 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:07.425672 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:07.425775 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:07.426443 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:07.426447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:07.426448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:17:07.426669 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:07.426708 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:17:07.426724 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:07.426833 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:07.426908 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 20:17:07.426937 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260807 20:17:07.427294 27134 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ca2a5b38ce4829b259a685ee93c3a5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42029 } }
I20260807 20:17:07.427382 27134 leader_election.cc:302] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0ca2a5b38ce4829b259a685ee93c3a5; no voters:
I20260807 20:17:07.427511 27134 leader_election.cc:288] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:17:07.427567 27140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:17:07.427681 27140 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [term 1 LEADER]: Becoming Leader. State: Replica: e0ca2a5b38ce4829b259a685ee93c3a5, State: Running, Role: LEADER
I20260807 20:17:07.427784 27134 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:17:07.427772 27140 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ca2a5b38ce4829b259a685ee93c3a5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42029 } }
I20260807 20:17:07.428130 27141 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0ca2a5b38ce4829b259a685ee93c3a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ca2a5b38ce4829b259a685ee93c3a5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42029 } } }
I20260807 20:17:07.428212 27141 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:07.428231 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:17:07.428427 27142 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0ca2a5b38ce4829b259a685ee93c3a5. Latest consensus state: current_term: 1 leader_uuid: "e0ca2a5b38ce4829b259a685ee93c3a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ca2a5b38ce4829b259a685ee93c3a5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 42029 } } }
I20260807 20:17:07.428731 27142 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5 [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:07.428887 27155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:17:07.429484 27155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:17:07.430953 27155 catalog_manager.cc:1347] Generated new cluster ID: 5c5fb24ae3d0415280481ee803af74a0
I20260807 20:17:07.431002 27155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:17:07.433347 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:07.429851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52795 (local address 127.23.200.254:42029)
0807 20:17:07.429884 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:07.429886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:07.429894 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:17:07.429967 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:07.429969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:07.429978 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:07.430008 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:07.430199 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:07.430890 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:07.431919 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:07.432240 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:07.432284 (+ 44us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:07.433043 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:07.433045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:07.433045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:17:07.433049 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:07.433062 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:17:07.433067 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:07.433131 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:07.433263 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 20:17:07.433275 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:17:07.446859 27155 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:17:07.447374 27155 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:17:07.449383 27155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0ca2a5b38ce4829b259a685ee93c3a5: Generated new TSK 0
I20260807 20:17:07.449568 27155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:17:07.458873 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 27028
W20260807 20:17:07.477308 27004 connection.cc:535] client connection to 127.23.200.254:42029 recv error: Network error: failed to read from TLS socket (remote: 127.23.200.254:42029): Connection reset by peer (error 104)
2026-08-07T20:17:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1455 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:17:07 dist-test-slave-2j7r krb5kdc[27192](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:17:07 dist-test-slave-2j7r krb5kdc[27192](info): set up 2 sockets
Aug 07 20:17:07 dist-test-slave-2j7r krb5kdc[27192](info): commencing operation
krb5kdc: starting...
W20260807 20:17:08.635608 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:17:08 dist-test-slave-2j7r krb5kdc[27192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:17:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:17:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:17:08.760638 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:36941
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:32971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:36941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:17:08.843209 27213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:17:08.843421 27213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:17:08.843453 27213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:17:08.844594 27213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:17:08.844636 27213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:17:08.844655 27213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:17:08.844669 27213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:17:08.844686 27213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:17:08.846258 27213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:32971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:36941
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:36941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:17:08.846503 27213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:17:08.846709 27213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:17:08.851548 27227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:08.851569 27222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:08.851548 27225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:08.851619 27213 server_base.cc:1034] running on GCE node
I20260807 20:17:08.852056 27213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:17:08.852344 27213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:17:08.853490 27213 hybrid_clock.cc:648] HybridClock initialized: now 1786133828853471 us; error 28 us; skew 500 ppm
Aug 07 20:17:08 dist-test-slave-2j7r krb5kdc[27192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133828, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:17:08.856021 27213 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:17:08.856475 27213 webserver.cc:460] Webserver started at http://127.23.200.254:46217/ using document root <none> and password file <none>
I20260807 20:17:08.856685 27213 fs_manager.cc:359] Metadata directory not provided
I20260807 20:17:08.856734 27213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:17:08.856851 27213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:17:08.857996 27213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "1d1635c5c651402ab63b44c17522cd9d"
format_stamp: "Formatted at 2026-08-07 20:17:08 on dist-test-slave-2j7r"
server_key: "1bf025d1fcbc168404625e820e4ee05d"
server_key_iv: "f2174909c0cf1b0adcb6af8999368541"
server_key_version: "encryptionkey@0"
I20260807 20:17:08.858326 27213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "1d1635c5c651402ab63b44c17522cd9d"
format_stamp: "Formatted at 2026-08-07 20:17:08 on dist-test-slave-2j7r"
server_key: "1bf025d1fcbc168404625e820e4ee05d"
server_key_iv: "f2174909c0cf1b0adcb6af8999368541"
server_key_version: "encryptionkey@0"
I20260807 20:17:08.859642 27213 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 20:17:08.860478 27238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:08.860690 27213 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:17:08.860803 27213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "1d1635c5c651402ab63b44c17522cd9d"
format_stamp: "Formatted at 2026-08-07 20:17:08 on dist-test-slave-2j7r"
server_key: "1bf025d1fcbc168404625e820e4ee05d"
server_key_iv: "f2174909c0cf1b0adcb6af8999368541"
server_key_version: "encryptionkey@0"
I20260807 20:17:08.860867 27213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:17:08.865424 27213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:17:08.868062 27213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:17:08.868266 27213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:17:08.872282 27213 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:36941
I20260807 20:17:08.872377 27317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:36941 every 8 connection(s)
I20260807 20:17:08.872658 27213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:17:08.873708 27319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:17:08.875981 27319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d: Bootstrap starting.
I20260807 20:17:08.876048 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 27213
I20260807 20:17:08.876157 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:17:08.876348 24355 external_mini_cluster.cc:1428] Setting key 31da0ffbd6963cae2e4874a82464ca77
I20260807 20:17:08.876593 27319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d: Neither blocks nor log segments found. Creating new log.
I20260807 20:17:08.876965 27319 log.cc:824] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d: Log is configured to *not* fsync() on all Append() calls
I20260807 20:17:08.877660 27319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d: No bootstrap required, opened a new log
I20260807 20:17:08.878731 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:08.876810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41011 (local address 127.23.200.254:36941)
0807 20:17:08.876992 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:08.877000 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:08.877027 (+ 27us) server_negotiation.cc:407] Connection header received
0807 20:17:08.877540 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:08.877548 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:08.877577 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:08.877701 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:08.878310 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:08.878314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:08.878314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:17:08.878318 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:08.878356 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:17:08.878367 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:08.878440 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:08.878497 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 20:17:08.878531 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
I20260807 20:17:08.879731 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:17:08.879941 27319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1635c5c651402ab63b44c17522cd9d" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 36941 } }
I20260807 20:17:08.880072 27319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:17:08.880093 27319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1635c5c651402ab63b44c17522cd9d, State: Initialized, Role: FOLLOWER
I20260807 20:17:08.880208 27319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1635c5c651402ab63b44c17522cd9d" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 36941 } }
I20260807 20:17:08.880280 27319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:17:08.880304 27319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:17:08.880349 27319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:17:08.880941 27319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1635c5c651402ab63b44c17522cd9d" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 36941 } }
I20260807 20:17:08.881052 27319 leader_election.cc:302] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d1635c5c651402ab63b44c17522cd9d; no voters:
I20260807 20:17:08.881264 27319 leader_election.cc:288] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:17:08.881297 27340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:17:08.881397 27340 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [term 1 LEADER]: Becoming Leader. State: Replica: 1d1635c5c651402ab63b44c17522cd9d, State: Running, Role: LEADER
I20260807 20:17:08.881517 27319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:17:08.882197 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:08.881105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46125 (local address 127.23.200.254:36941)
0807 20:17:08.881139 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:08.881141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:08.881322 (+ 181us) server_negotiation.cc:407] Connection header received
0807 20:17:08.881396 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:08.881397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:08.881402 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:08.881438 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:08.882010 (+ 572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:08.882011 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:08.882012 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:17:08.882016 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:08.882029 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:17:08.882033 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:08.882078 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:08.882126 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 20:17:08.882138 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 20:17:08.881518 27340 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1635c5c651402ab63b44c17522cd9d" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 36941 } }
I20260807 20:17:08.883263 27343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d1635c5c651402ab63b44c17522cd9d. Latest consensus state: current_term: 1 leader_uuid: "1d1635c5c651402ab63b44c17522cd9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1635c5c651402ab63b44c17522cd9d" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 36941 } } }
I20260807 20:17:08.883322 27343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:08.883141 27341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d1635c5c651402ab63b44c17522cd9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d1635c5c651402ab63b44c17522cd9d" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 36941 } } }
I20260807 20:17:08.883471 27341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d [sys.catalog]: This master's current role is: LEADER
W20260807 20:17:08.883783 27364 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:17:08.883846 27364 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:17:08.883903 27366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:17:08.884076 27366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:17:08.885439 27366 catalog_manager.cc:1347] Generated new cluster ID: 306f25913f6f40c6970e26fabf245ed7
I20260807 20:17:08.885494 27366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:17:08.896829 27366 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:17:08.897002 27366 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:17:08.903538 27366 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d1635c5c651402ab63b44c17522cd9d: Generated new TSK 0
I20260807 20:17:08.903788 27366 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:17:08.905911 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 27213
2026-08-07T20:17:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1447 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:17:08 dist-test-slave-2j7r krb5kdc[27379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:17:08 dist-test-slave-2j7r krb5kdc[27379](info): set up 2 sockets
Aug 07 20:17:08 dist-test-slave-2j7r krb5kdc[27379](info): commencing operation
krb5kdc: starting...
W20260807 20:17:10.085378 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:17:10 dist-test-slave-2j7r krb5kdc[27379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:17:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:17:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:17:10.211977 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:39527
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:36079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:39527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:17:10.296756 27396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:17:10.296988 27396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:17:10.297011 27396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:17:10.298228 27396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:17:10.298285 27396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:17:10.298300 27396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:17:10.298313 27396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:17:10.298323 27396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:17:10.299998 27396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:36079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:39527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:39527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:17:10.300261 27396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:17:10.300490 27396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:17:10.305420 27403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:10.305521 27396 server_base.cc:1034] running on GCE node
W20260807 20:17:10.305473 27406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:10.305488 27410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:10.305902 27396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:17:10.306237 27396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:17:10.307384 27396 hybrid_clock.cc:648] HybridClock initialized: now 1786133830307370 us; error 31 us; skew 500 ppm
Aug 07 20:17:10 dist-test-slave-2j7r krb5kdc[27379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133830, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:17:10.309943 27396 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:17:10.310387 27396 webserver.cc:460] Webserver started at http://127.23.200.254:43627/ using document root <none> and password file <none>
I20260807 20:17:10.310606 27396 fs_manager.cc:359] Metadata directory not provided
I20260807 20:17:10.310654 27396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:17:10.310782 27396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:17:10.311863 27396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "03ef333be25d4d6c9435aba32d24038c"
format_stamp: "Formatted at 2026-08-07 20:17:10 on dist-test-slave-2j7r"
server_key: "769851197472a271b8391f9377563f20"
server_key_iv: "c58b0fe8dcfaf55355ecbd184106fd40"
server_key_version: "encryptionkey@0"
I20260807 20:17:10.312213 27396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "03ef333be25d4d6c9435aba32d24038c"
format_stamp: "Formatted at 2026-08-07 20:17:10 on dist-test-slave-2j7r"
server_key: "769851197472a271b8391f9377563f20"
server_key_iv: "c58b0fe8dcfaf55355ecbd184106fd40"
server_key_version: "encryptionkey@0"
I20260807 20:17:10.313549 27396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:17:10.314248 27422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:10.314425 27396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:10.314520 27396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "03ef333be25d4d6c9435aba32d24038c"
format_stamp: "Formatted at 2026-08-07 20:17:10 on dist-test-slave-2j7r"
server_key: "769851197472a271b8391f9377563f20"
server_key_iv: "c58b0fe8dcfaf55355ecbd184106fd40"
server_key_version: "encryptionkey@0"
I20260807 20:17:10.314579 27396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:17:10.333930 27396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:17:10.335870 27396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:17:10.336016 27396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:17:10.340214 27396 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:39527
I20260807 20:17:10.340257 27513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:39527 every 8 connection(s)
I20260807 20:17:10.340615 27396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:17:10.341552 27514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:17:10.344142 27514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c: Bootstrap starting.
I20260807 20:17:10.344795 27514 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c: Neither blocks nor log segments found. Creating new log.
I20260807 20:17:10.345034 27514 log.cc:824] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c: Log is configured to *not* fsync() on all Append() calls
I20260807 20:17:10.345715 27514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c: No bootstrap required, opened a new log
I20260807 20:17:10.347270 27514 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ef333be25d4d6c9435aba32d24038c" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39527 } }
I20260807 20:17:10.347411 27514 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:17:10.347441 27514 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03ef333be25d4d6c9435aba32d24038c, State: Initialized, Role: FOLLOWER
I20260807 20:17:10.347551 27514 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ef333be25d4d6c9435aba32d24038c" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39527 } }
I20260807 20:17:10.347604 27514 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:17:10.347623 27514 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:17:10.347647 27514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:17:10.347877 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 27396
I20260807 20:17:10.347985 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:17:10.348181 24355 external_mini_cluster.cc:1428] Setting key 5cb27b335e58885b921335b95d7c150a
I20260807 20:17:10.348672 27514 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ef333be25d4d6c9435aba32d24038c" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39527 } }
I20260807 20:17:10.348785 27514 leader_election.cc:302] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03ef333be25d4d6c9435aba32d24038c; no voters:
I20260807 20:17:10.348968 27514 leader_election.cc:288] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:17:10.349004 27522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:17:10.349120 27522 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [term 1 LEADER]: Becoming Leader. State: Replica: 03ef333be25d4d6c9435aba32d24038c, State: Running, Role: LEADER
I20260807 20:17:10.349210 27522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ef333be25d4d6c9435aba32d24038c" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39527 } }
I20260807 20:17:10.349257 27514 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:17:10.349682 27525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03ef333be25d4d6c9435aba32d24038c. Latest consensus state: current_term: 1 leader_uuid: "03ef333be25d4d6c9435aba32d24038c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ef333be25d4d6c9435aba32d24038c" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39527 } } }
I20260807 20:17:10.349759 27525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:10.350023 27535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:17:10.350445 27524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03ef333be25d4d6c9435aba32d24038c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ef333be25d4d6c9435aba32d24038c" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39527 } } }
I20260807 20:17:10.350544 27535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:17:10.350605 27524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:10.352084 27535 catalog_manager.cc:1347] Generated new cluster ID: 37a4d94673c6448cb4bf0cfca5278676
I20260807 20:17:10.352137 27535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:17:10 dist-test-slave-2j7r krb5kdc[27379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:17:10.359083 27519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:10.348694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39715 (local address 127.23.200.254:39527)
0807 20:17:10.348867 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:10.348877 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:10.348920 (+ 43us) server_negotiation.cc:407] Connection header received
0807 20:17:10.349047 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:10.349055 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:10.349139 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:10.349317 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:10.350802 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:10.352027 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:10.352776 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:10.353238 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:10.353341 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:10.355552 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:10.355556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:10.355557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:10.355569 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:10.357610 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:10.357925 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:10.357929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:10.357929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:10.358025 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:10.358207 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:10.358208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:10.358208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:10.358470 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:10.358607 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:10.358743 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 20:17:10.358881 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260807 20:17:10.360020 27535 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:17:10.360184 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:17:10.360606 27535 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:17:10.368616 27519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:10.361732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37795 (local address 127.23.200.254:39527)
0807 20:17:10.361768 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:10.361770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:10.361917 (+ 147us) server_negotiation.cc:407] Connection header received
0807 20:17:10.361944 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:10.361945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:10.361956 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:10.362021 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:10.362587 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:10.363359 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:10.364330 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:10.364665 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:10.364725 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:10.366031 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:10.366032 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:10.366033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:10.366040 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:10.367520 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:10.367870 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:10.367871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:10.367872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:10.367933 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:10.368103 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:10.368104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:10.368104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:10.368227 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:10.368320 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:10.368437 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 20:17:10.368518 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":18}
I20260807 20:17:10.375131 27535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03ef333be25d4d6c9435aba32d24038c: Generated new TSK 0
I20260807 20:17:10.375319 27535 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:17:10.379825 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 27396
2026-08-07T20:17:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1470 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:17:10 dist-test-slave-2j7r krb5kdc[27569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:17:10 dist-test-slave-2j7r krb5kdc[27569](info): set up 2 sockets
Aug 07 20:17:10 dist-test-slave-2j7r krb5kdc[27569](info): commencing operation
krb5kdc: starting...
W20260807 20:17:11.551294 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:17:11 dist-test-slave-2j7r krb5kdc[27569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:17:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:17:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:17:11.676043 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:43861
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:43861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:17:11.759496 27593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:17:11.759696 27593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:17:11.759725 27593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:17:11.760869 27593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:17:11.760908 27593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:17:11.760927 27593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:17:11.760944 27593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:17:11.760959 27593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:17:11.762503 27593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:43861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:43861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:17:11.762758 27593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:17:11.762971 27593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:17:11.767779 27605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:11.767827 27602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:11.767794 27599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:11.767827 27593 server_base.cc:1034] running on GCE node
I20260807 20:17:11.768177 27593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:17:11.768445 27593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:17:11.769587 27593 hybrid_clock.cc:648] HybridClock initialized: now 1786133831769572 us; error 31 us; skew 500 ppm
Aug 07 20:17:11 dist-test-slave-2j7r krb5kdc[27569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133831, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:17:11.778147 27593 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:17:11.778577 27593 webserver.cc:460] Webserver started at http://127.23.200.254:41961/ using document root <none> and password file <none>
I20260807 20:17:11.778795 27593 fs_manager.cc:359] Metadata directory not provided
I20260807 20:17:11.778844 27593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:17:11.778980 27593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:17:11.780040 27593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "f302d78afd614ce09692edcfbeebe674"
format_stamp: "Formatted at 2026-08-07 20:17:11 on dist-test-slave-2j7r"
server_key: "d62512c1ce37a695b1e133373abcdd5b"
server_key_iv: "648be060c3abf4dba1a50e90e9e7d6c2"
server_key_version: "encryptionkey@0"
I20260807 20:17:11.780372 27593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "f302d78afd614ce09692edcfbeebe674"
format_stamp: "Formatted at 2026-08-07 20:17:11 on dist-test-slave-2j7r"
server_key: "d62512c1ce37a695b1e133373abcdd5b"
server_key_iv: "648be060c3abf4dba1a50e90e9e7d6c2"
server_key_version: "encryptionkey@0"
I20260807 20:17:11.781810 27593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:17:11.782534 27617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:11.782692 27593 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:17:11.782783 27593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "f302d78afd614ce09692edcfbeebe674"
format_stamp: "Formatted at 2026-08-07 20:17:11 on dist-test-slave-2j7r"
server_key: "d62512c1ce37a695b1e133373abcdd5b"
server_key_iv: "648be060c3abf4dba1a50e90e9e7d6c2"
server_key_version: "encryptionkey@0"
I20260807 20:17:11.782842 27593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:17:11.790346 27593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:17:11.792520 27593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:17:11.792656 27593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:17:11.796556 27593 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:43861
I20260807 20:17:11.796597 27693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:43861 every 8 connection(s)
I20260807 20:17:11.796954 27593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:17:11.797875 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:17:11.800165 27694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674: Bootstrap starting.
I20260807 20:17:11.800751 27694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674: Neither blocks nor log segments found. Creating new log.
I20260807 20:17:11.801003 27694 log.cc:824] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674: Log is configured to *not* fsync() on all Append() calls
I20260807 20:17:11.801749 27694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674: No bootstrap required, opened a new log
I20260807 20:17:11.801764 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 27593
I20260807 20:17:11.801867 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:17:11.802037 24355 external_mini_cluster.cc:1428] Setting key fc0f38ebe41d8cbf9bcb191d1096f771
I20260807 20:17:11.803813 27694 raft_consensus.cc:348] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f302d78afd614ce09692edcfbeebe674" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 43861 } }
I20260807 20:17:11.803992 27694 raft_consensus.cc:374] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:17:11.804029 27694 raft_consensus.cc:729] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f302d78afd614ce09692edcfbeebe674, State: Initialized, Role: FOLLOWER
I20260807 20:17:11.804178 27694 consensus_queue.cc:260] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f302d78afd614ce09692edcfbeebe674" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 43861 } }
I20260807 20:17:11.804265 27694 raft_consensus.cc:388] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:17:11.804297 27694 raft_consensus.cc:482] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:17:11.804345 27694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:17:11.805051 27694 raft_consensus.cc:504] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f302d78afd614ce09692edcfbeebe674" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 43861 } }
I20260807 20:17:11.805186 27694 leader_election.cc:302] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f302d78afd614ce09692edcfbeebe674; no voters:
I20260807 20:17:11.805333 27694 leader_election.cc:288] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:17:11.805366 27702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:17:11.805531 27702 raft_consensus.cc:686] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [term 1 LEADER]: Becoming Leader. State: Replica: f302d78afd614ce09692edcfbeebe674, State: Running, Role: LEADER
I20260807 20:17:11.805567 27694 sys_catalog.cc:564] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:17:11.805683 27702 consensus_queue.cc:237] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f302d78afd614ce09692edcfbeebe674" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 43861 } }
I20260807 20:17:11.806037 27702 sys_catalog.cc:455] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f302d78afd614ce09692edcfbeebe674. Latest consensus state: current_term: 1 leader_uuid: "f302d78afd614ce09692edcfbeebe674" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f302d78afd614ce09692edcfbeebe674" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 43861 } } }
I20260807 20:17:11.806217 27702 sys_catalog.cc:458] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:11.806164 27704 sys_catalog.cc:455] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f302d78afd614ce09692edcfbeebe674" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f302d78afd614ce09692edcfbeebe674" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 43861 } } }
I20260807 20:17:11.806464 27704 sys_catalog.cc:458] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674 [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:11.806469 27718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:17:11.807041 27718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:17:11.808871 27701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:11.802593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34237 (local address 127.23.200.254:43861)
0807 20:17:11.802791 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:11.802801 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:11.802845 (+ 44us) server_negotiation.cc:407] Connection header received
0807 20:17:11.803766 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:11.803776 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:11.803842 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:11.803996 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:11.804223 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:11.805906 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:11.806995 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:11.807543 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:11.807647 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:11.808272 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:11.808276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:11.808277 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:17:11.808444 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:11.808476 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:17:11.808487 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:11.808546 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:11.808620 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 20:17:11.808685 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260807 20:17:11.809821 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:17:11.813611 27701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:11.811090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52539 (local address 127.23.200.254:43861)
0807 20:17:11.811124 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:11.811126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:11.811239 (+ 113us) server_negotiation.cc:407] Connection header received
0807 20:17:11.811296 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:11.811297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:11.811307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:11.811346 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:11.811453 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:11.811900 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:11.812542 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:11.812721 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:11.812773 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:11.813378 (+ 605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:11.813379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:11.813379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:17:11.813385 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:11.813400 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:17:11.813405 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:11.813466 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:11.813522 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 20:17:11.813545 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 20:17:11.814323 27718 catalog_manager.cc:1347] Generated new cluster ID: 76dd015ddd5d4d04bdc6e008d3dc460e
I20260807 20:17:11.814380 27718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:17:11.820724 27718 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:17:11.821255 27718 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:17:11.825471 27718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f302d78afd614ce09692edcfbeebe674: Generated new TSK 0
I20260807 20:17:11.825637 27718 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:17:11.827838 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 27593
2026-08-07T20:17:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1447 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:17:11 dist-test-slave-2j7r krb5kdc[27748](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:17:11 dist-test-slave-2j7r krb5kdc[27748](info): set up 2 sockets
Aug 07 20:17:11 dist-test-slave-2j7r krb5kdc[27748](info): commencing operation
krb5kdc: starting...
W20260807 20:17:13.002614 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:17:13 dist-test-slave-2j7r krb5kdc[27748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:17:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:17:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:17:13.123550 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:39709
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:39709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:17:13.207110 27765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:17:13.207338 27765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:17:13.207372 27765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:17:13.208559 27765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:17:13.208606 27765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:17:13.208624 27765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:17:13.208640 27765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:17:13.208655 27765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:17:13.210232 27765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:39709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:39709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:17:13.210516 27765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:17:13.210731 27765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:17:13.215638 27774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:13.215662 27779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:13.215638 27775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:13.215739 27765 server_base.cc:1034] running on GCE node
I20260807 20:17:13.216127 27765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:17:13.216471 27765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:17:13.217614 27765 hybrid_clock.cc:648] HybridClock initialized: now 1786133833217592 us; error 34 us; skew 500 ppm
Aug 07 20:17:13 dist-test-slave-2j7r krb5kdc[27748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133833, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:17:13.220039 27765 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:17:13.220522 27765 webserver.cc:460] Webserver started at http://127.23.200.254:46871/ using document root <none> and password file <none>
I20260807 20:17:13.220748 27765 fs_manager.cc:359] Metadata directory not provided
I20260807 20:17:13.220796 27765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:17:13.220927 27765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:17:13.222023 27765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "fb7f2d27329e4649b897702e346cdc87"
format_stamp: "Formatted at 2026-08-07 20:17:13 on dist-test-slave-2j7r"
server_key: "eae70725911ec51d1a9a8dcc7af1693b"
server_key_iv: "cfc57b569ef29024b779f33b474b6ec8"
server_key_version: "encryptionkey@0"
I20260807 20:17:13.222361 27765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "fb7f2d27329e4649b897702e346cdc87"
format_stamp: "Formatted at 2026-08-07 20:17:13 on dist-test-slave-2j7r"
server_key: "eae70725911ec51d1a9a8dcc7af1693b"
server_key_iv: "cfc57b569ef29024b779f33b474b6ec8"
server_key_version: "encryptionkey@0"
I20260807 20:17:13.223665 27765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:17:13.224331 27788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:13.224486 27765 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:17:13.224578 27765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "fb7f2d27329e4649b897702e346cdc87"
format_stamp: "Formatted at 2026-08-07 20:17:13 on dist-test-slave-2j7r"
server_key: "eae70725911ec51d1a9a8dcc7af1693b"
server_key_iv: "cfc57b569ef29024b779f33b474b6ec8"
server_key_version: "encryptionkey@0"
I20260807 20:17:13.224646 27765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:17:13.228010 27765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:17:13.230115 27765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:17:13.230249 27765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:17:13.234185 27765 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:39709
I20260807 20:17:13.234304 27864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:39709 every 8 connection(s)
I20260807 20:17:13.234560 27765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:17:13.235617 27865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:17:13.237915 27865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87: Bootstrap starting.
I20260807 20:17:13.238447 27865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87: Neither blocks nor log segments found. Creating new log.
I20260807 20:17:13.238700 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 27765
I20260807 20:17:13.238760 27865 log.cc:824] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87: Log is configured to *not* fsync() on all Append() calls
I20260807 20:17:13.238814 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:17:13.238994 24355 external_mini_cluster.cc:1428] Setting key c0cd2d0fbb34ef3730b0a7e650db4311
I20260807 20:17:13.239559 27865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87: No bootstrap required, opened a new log
I20260807 20:17:13.241397 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:13.239493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34201 (local address 127.23.200.254:39709)
0807 20:17:13.239656 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:13.239665 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:13.239696 (+ 31us) server_negotiation.cc:407] Connection header received
0807 20:17:13.240138 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:13.240147 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:13.240181 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:13.240328 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:13.240877 (+ 549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:13.240881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:13.240882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:17:13.240886 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:13.240935 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:17:13.240950 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:13.241039 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:13.241161 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 20:17:13.241203 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260807 20:17:13.242123 27865 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7f2d27329e4649b897702e346cdc87" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39709 } }
I20260807 20:17:13.242254 27865 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:17:13.242285 27865 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb7f2d27329e4649b897702e346cdc87, State: Initialized, Role: FOLLOWER
I20260807 20:17:13.242280 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:17:13.242412 27865 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7f2d27329e4649b897702e346cdc87" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39709 } }
I20260807 20:17:13.242501 27865 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:17:13.242542 27865 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:17:13.242590 27865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:17:13.243163 27865 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7f2d27329e4649b897702e346cdc87" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39709 } }
I20260807 20:17:13.243263 27865 leader_election.cc:302] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb7f2d27329e4649b897702e346cdc87; no voters:
I20260807 20:17:13.243428 27865 leader_election.cc:288] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:17:13.243461 27876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:17:13.243639 27865 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:17:13.243956 27876 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [term 1 LEADER]: Becoming Leader. State: Replica: fb7f2d27329e4649b897702e346cdc87, State: Running, Role: LEADER
I20260807 20:17:13.244088 27876 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7f2d27329e4649b897702e346cdc87" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39709 } }
I20260807 20:17:13.244768 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:13.243763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40345 (local address 127.23.200.254:39709)
0807 20:17:13.243806 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:13.243808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:13.243817 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:17:13.243916 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:13.243917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:13.243922 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:13.243949 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:13.244548 (+ 599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:13.244549 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:13.244550 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:17:13.244554 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:13.244568 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:17:13.244572 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:13.244615 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:13.244694 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 20:17:13.244707 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 20:17:13.244831 27878 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb7f2d27329e4649b897702e346cdc87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7f2d27329e4649b897702e346cdc87" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39709 } } }
I20260807 20:17:13.244911 27878 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:13.245132 27881 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb7f2d27329e4649b897702e346cdc87. Latest consensus state: current_term: 1 leader_uuid: "fb7f2d27329e4649b897702e346cdc87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7f2d27329e4649b897702e346cdc87" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 39709 } } }
I20260807 20:17:13.245144 27900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:17:13.245205 27881 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87 [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:13.245894 27900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:17:13.247253 27900 catalog_manager.cc:1347] Generated new cluster ID: e1f0ad7319424c58864cd6ba62e1ec0d
I20260807 20:17:13.247303 27900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:17:13.266074 27900 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:17:13.266251 27900 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:17:13.270177 27900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb7f2d27329e4649b897702e346cdc87: Generated new TSK 0
I20260807 20:17:13.270335 27900 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:17:13.277741 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 27765
2026-08-07T20:17:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1455 ms)
[----------] 18 tests from AuthTokenIssuingTest (26182 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:17:13 dist-test-slave-2j7r krb5kdc[27916](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:17:13 dist-test-slave-2j7r krb5kdc[27916](info): set up 2 sockets
Aug 07 20:17:13 dist-test-slave-2j7r krb5kdc[27916](info): commencing operation
krb5kdc: starting...
W20260807 20:17:14.442852 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:17:14 dist-test-slave-2j7r krb5kdc[27916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:17:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:17:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:17:14.563983 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:37141
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:44201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:37141,127.23.200.253:34859,127.23.200.252:41403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:17:14.645912 27943 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 20:17:14.646194 27943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:17:14.646219 27943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:17:14.646255 27943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:17:14.647684 27943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:17:14.647760 27943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:17:14.647787 27943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:17:14.647806 27943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:17:14.647831 27943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:17:14.649538 27943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:37141,127.23.200.253:34859,127.23.200.252:41403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:37141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:17:14.649798 27943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:17:14.650018 27943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:17:14.655087 27953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:14.655182 27943 server_base.cc:1034] running on GCE node
W20260807 20:17:14.655083 27956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:14.655292 27954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:14.655544 27943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:17:14.655797 27943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:17:14.656944 27943 hybrid_clock.cc:648] HybridClock initialized: now 1786133834656930 us; error 29 us; skew 500 ppm
Aug 07 20:17:14 dist-test-slave-2j7r krb5kdc[27916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:17:14.659351 27943 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:17:14.659744 27943 webserver.cc:460] Webserver started at http://127.23.200.254:44999/ using document root <none> and password file <none>
I20260807 20:17:14.659942 27943 fs_manager.cc:359] Metadata directory not provided
I20260807 20:17:14.659989 27943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:17:14.660107 27943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:17:14.661216 27943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "09d944951d9c455f8c5cd2f42fcabe08"
format_stamp: "Formatted at 2026-08-07 20:17:14 on dist-test-slave-2j7r"
server_key: "e48a66a53296deeb9d5dc4f402c89049"
server_key_iv: "3ceda59c43340d114c9f67f0c7371ce2"
server_key_version: "encryptionkey@0"
I20260807 20:17:14.661545 27943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "09d944951d9c455f8c5cd2f42fcabe08"
format_stamp: "Formatted at 2026-08-07 20:17:14 on dist-test-slave-2j7r"
server_key: "e48a66a53296deeb9d5dc4f402c89049"
server_key_iv: "3ceda59c43340d114c9f67f0c7371ce2"
server_key_version: "encryptionkey@0"
I20260807 20:17:14.662768 27943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:17:14.663547 27970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:14.663705 27943 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:17:14.663791 27943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "09d944951d9c455f8c5cd2f42fcabe08"
format_stamp: "Formatted at 2026-08-07 20:17:14 on dist-test-slave-2j7r"
server_key: "e48a66a53296deeb9d5dc4f402c89049"
server_key_iv: "3ceda59c43340d114c9f67f0c7371ce2"
server_key_version: "encryptionkey@0"
I20260807 20:17:14.663848 27943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:17:14.678429 27943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:17:14.680341 27943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:17:14.680485 27943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:17:14.684396 27943 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:37141
I20260807 20:17:14.684459 28052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:37141 every 8 connection(s)
I20260807 20:17:14.684777 27943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:17:14.685674 28054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:17:14.687245 28054 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:17:14.689003 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 27943
I20260807 20:17:14.689134 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:17:14.689316 24355 external_mini_cluster.cc:1428] Setting key cea04c8f18bcf4c1b777eede28e2ba63
Aug 07 20:17:14 dist-test-slave-2j7r krb5kdc[27916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:17:14.696734 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.688034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37034 (local address 127.23.200.254:37141)
0807 20:17:14.688149 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:14.688153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:14.688181 (+ 28us) server_negotiation.cc:407] Connection header received
0807 20:17:14.688638 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:14.688640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:14.688655 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:14.688728 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:14.689492 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.690308 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.691373 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.691843 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.691950 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:14.693952 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:14.693959 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:14.693960 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:14.693967 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:14.695337 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:14.695668 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:14.695672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:14.695672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:14.695746 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:14.695939 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:14.695940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:14.695940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:14.696139 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:14.696208 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:14.696385 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 20:17:14.696489 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":48,"threads_started":1}
I20260807 20:17:14.696738 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.687875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37141 (local address 127.0.0.1:37034)
0807 20:17:14.688029 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.688068 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:14.688521 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:14.688743 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:14.688746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:14.689119 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:14.689345 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:14.689351 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.690412 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.690413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:14.691295 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:14.691298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.691918 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.691919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:14.691976 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:14.692447 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:14.692456 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:14.693875 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:14.695413 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:14.695419 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:14.695422 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:14.695599 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:14.695813 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:14.695814 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:14.695814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:14.695886 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:14.696199 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:14.696204 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:14.696264 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 20:17:14.696346 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 20:17:14.696563 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":82,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for kudu/127.23.200.253@KRBTEST.COM; defaulting to no policy
I20260807 20:17:14.697445 28054 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:17:14.697757 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.697652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:34859 (local address 127.0.0.1:50960)
0807 20:17:14.697691 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.697734 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 20:17:14.697876 27976 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.200.253:34859: connect: Connection refused (error 111)
I20260807 20:17:14.698262 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.698182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:34859 (local address 127.0.0.1:50970)
0807 20:17:14.698213 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.698237 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260807 20:17:14.698354 28054 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.253:34859: Network error: Client connection negotiation failed: client connection to 127.23.200.253:34859: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.253.keytab.
Entry for principal kudu/127.23.200.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.253.keytab.
I20260807 20:17:14.715590 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.253:34859
--webserver_interface=127.23.200.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.253.keytab
--principal=kudu/127.23.200.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:44201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:37141,127.23.200.253:34859,127.23.200.252:41403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:17:14.747558 28054 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } attempt: 1
I20260807 20:17:14.748078 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.747945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:34859 (local address 127.0.0.1:50980)
0807 20:17:14.748006 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.748048 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 20:17:14.748478 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.748357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:34859 (local address 127.0.0.1:50992)
0807 20:17:14.748410 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.748453 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 20:17:14.748592 28054 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.253:34859: Network error: Client connection negotiation failed: client connection to 127.23.200.253:34859: connect: Connection refused (error 111)
W20260807 20:17:14.798009 28066 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 20:17:14.798236 28066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:17:14.798262 28066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:17:14.798288 28066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:17:14.799437 28066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:17:14.799477 28066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:17:14.799495 28066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:17:14.799510 28066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:17:14.799525 28066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:17:14.801007 28066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:37141,127.23.200.253:34859,127.23.200.252:41403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.253.keytab
--principal=kudu/127.23.200.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.253:34859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.200.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:17:14.801266 28066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:17:14.801470 28066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:17:14.806354 28066 server_base.cc:1034] running on GCE node
W20260807 20:17:14.806305 28078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:14.806316 28074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:14.806371 28075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:14.806771 28066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:17:14.807060 28066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:17:14.808211 28066 hybrid_clock.cc:648] HybridClock initialized: now 1786133834808195 us; error 29 us; skew 500 ppm
Aug 07 20:17:14 dist-test-slave-2j7r krb5kdc[27916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:17:14.810577 28066 init.cc:377] Logged in from keytab as kudu/127.23.200.253@KRBTEST.COM (short username kudu)
I20260807 20:17:14.811003 28066 webserver.cc:460] Webserver started at http://127.23.200.253:38261/ using document root <none> and password file <none>
I20260807 20:17:14.811199 28066 fs_manager.cc:359] Metadata directory not provided
I20260807 20:17:14.811240 28066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:17:14.811362 28066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:17:14.812389 28066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/data/instance:
uuid: "a6f0e6bf2fe1430190fb9364ce2030f9"
format_stamp: "Formatted at 2026-08-07 20:17:14 on dist-test-slave-2j7r"
server_key: "4f6b7ec592286f5a5eb3f2bad3293f57"
server_key_iv: "58deea77d38c4fb6a43caa2e3af61929"
server_key_version: "encryptionkey@0"
I20260807 20:17:14.812724 28066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/wal/instance:
uuid: "a6f0e6bf2fe1430190fb9364ce2030f9"
format_stamp: "Formatted at 2026-08-07 20:17:14 on dist-test-slave-2j7r"
server_key: "4f6b7ec592286f5a5eb3f2bad3293f57"
server_key_iv: "58deea77d38c4fb6a43caa2e3af61929"
server_key_version: "encryptionkey@0"
I20260807 20:17:14.814025 28066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:17:14.814678 28088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:14.814822 28066 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:17:14.814903 28066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/wal
uuid: "a6f0e6bf2fe1430190fb9364ce2030f9"
format_stamp: "Formatted at 2026-08-07 20:17:14 on dist-test-slave-2j7r"
server_key: "4f6b7ec592286f5a5eb3f2bad3293f57"
server_key_iv: "58deea77d38c4fb6a43caa2e3af61929"
server_key_version: "encryptionkey@0"
I20260807 20:17:14.814985 28066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:17:14.816756 28054 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } attempt: 2
I20260807 20:17:14.817183 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.817029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:34859 (local address 127.0.0.1:50996)
0807 20:17:14.817104 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.817145 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 20:17:14.817536 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.817446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:34859 (local address 127.0.0.1:51004)
0807 20:17:14.817484 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.817511 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:34859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 20:17:14.817620 28054 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.253:34859: Network error: Client connection negotiation failed: client connection to 127.23.200.253:34859: connect: Connection refused (error 111)
I20260807 20:17:14.830444 28066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:17:14.832157 28066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:17:14.832268 28066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:17:14.835959 28066 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.253:34859
I20260807 20:17:14.835994 28170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.253:34859 every 8 connection(s)
I20260807 20:17:14.836302 28066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/data/info.pb
I20260807 20:17:14.837167 28172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:17:14.838677 28172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:17:14.841077 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 28066
I20260807 20:17:14.841188 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-1/wal/instance
I20260807 20:17:14.841307 24355 external_mini_cluster.cc:1428] Setting key 654154efb80245707499d890f903157d
Aug 07 20:17:14 dist-test-slave-2j7r krb5kdc[27916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.253@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:17:14.846940 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.839318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37044 (local address 127.23.200.254:37141)
0807 20:17:14.839392 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:14.839394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:14.839553 (+ 159us) server_negotiation.cc:407] Connection header received
0807 20:17:14.839998 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:14.839999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:14.840012 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:14.840061 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:14.840603 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.841063 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.842056 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.842257 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.842311 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:14.844436 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:14.844438 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:14.844438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:14.844445 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:14.845658 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:14.846024 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:14.846025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:14.846025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:14.846076 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:14.846305 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:14.846306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:14.846307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:14.846443 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:14.846477 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:14.846753 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 20:17:14.846838 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 20:17:14.847055 28175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.839290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37141 (local address 127.0.0.1:37044)
0807 20:17:14.839442 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.839476 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:14.839902 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:14.840123 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:14.840127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:14.840351 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:14.840542 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:14.840545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.841179 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.841181 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:14.841954 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:14.841957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.842326 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.842327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:14.842464 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:14.842920 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:14.842941 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:14.844351 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:14.845728 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:14.845735 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:14.845737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:14.845946 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:14.846128 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:14.846129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:14.846130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:14.846228 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:14.846519 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:14.846525 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:14.846635 (+ 110us) client_negotiation.cc:769] Sending connection context
0807 20:17:14.846715 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 20:17:14.846855 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
I20260807 20:17:14.847606 28172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.200.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.252@KRBTEST.COM" created.
Aug 07 20:17:14 dist-test-slave-2j7r krb5kdc[27916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.253@KRBTEST.COM for kudu/127.23.200.253@KRBTEST.COM
I20260807 20:17:14.856063 28175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.847857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:34859 (local address 127.0.0.1:51008)
0807 20:17:14.847903 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.847915 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:14.847966 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:14.848197 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:14.848198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:14.848330 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:14.848404 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:14.848406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.849240 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.849241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:14.850019 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:14.850022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.850468 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.850469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:14.850526 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:14.850892 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:14.850897 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:14.853293 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:14.854916 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:14.854917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:14.854917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:14.855115 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:14.855391 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:14.855392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:14.855392 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:14.855490 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:14.855823 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:14.855823 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:14.855856 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 20:17:14.855920 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 20:17:14.855998 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 20:17:14.856160 28177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.847935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51008 (local address 127.23.200.253:34859)
0807 20:17:14.848067 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:14.848072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:14.848090 (+ 18us) server_negotiation.cc:407] Connection header received
0807 20:17:14.848109 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:14.848110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:14.848120 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:14.848178 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:14.848477 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.849177 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.850099 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.850388 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.850455 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:14.853371 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:14.853381 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:14.853381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:14.853389 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:14.854830 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:14.855215 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:14.855218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:14.855219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:14.855281 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:14.855575 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:14.855576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:14.855576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:14.855754 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:14.855872 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:14.855939 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 20:17:14.856065 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260807 20:17:14.856865 28172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:17:14.857211 28175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.857072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:41403 (local address 127.0.0.1:33468)
0807 20:17:14.857137 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.857189 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 20:17:14.857347 28091 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.200.252:41403: connect: Connection refused (error 111)
I20260807 20:17:14.857739 28175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.857640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:41403 (local address 127.0.0.1:33480)
0807 20:17:14.857685 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.857717 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 20:17:14.857847 28172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.252:41403: Network error: Client connection negotiation failed: client connection to 127.23.200.252:41403: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.252.keytab.
Entry for principal kudu/127.23.200.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.252.keytab.
I20260807 20:17:14.868710 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.252:41403
--webserver_interface=127.23.200.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.252.keytab
--principal=kudu/127.23.200.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:44201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:37141,127.23.200.253:34859,127.23.200.252:41403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:17:14.907085 28172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } attempt: 1
I20260807 20:17:14.907604 28175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.907463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:41403 (local address 127.0.0.1:33494)
0807 20:17:14.907524 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.907569 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 20:17:14.907950 28175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.907864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:41403 (local address 127.0.0.1:33498)
0807 20:17:14.907902 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.907931 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 20:17:14.908051 28172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.252:41403: Network error: Client connection negotiation failed: client connection to 127.23.200.252:41403: connect: Connection refused (error 111)
I20260807 20:17:14.908747 28054 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } attempt: 3
Aug 07 20:17:14 dist-test-slave-2j7r krb5kdc[27916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for kudu/127.23.200.253@KRBTEST.COM
I20260807 20:17:14.914940 28177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.909051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51010 (local address 127.23.200.253:34859)
0807 20:17:14.909127 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:14.909129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:14.909139 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:17:14.909184 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:14.909185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:14.909198 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:14.909244 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:14.909552 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.910028 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.910730 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.910940 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.910991 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:14.912725 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:14.912726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:14.912727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:14.912734 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:14.913996 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:14.914284 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:14.914285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:14.914286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:14.914333 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:14.914494 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:14.914495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:14.914495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:14.914598 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:14.914654 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:14.914783 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 20:17:14.914852 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 20:17:14.914953 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.908998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:34859 (local address 127.0.0.1:51010)
0807 20:17:14.909051 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.909062 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:14.909130 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:14.909254 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:14.909255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:14.909414 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:14.909482 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:14.909484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.910100 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.910102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:14.910653 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:14.910655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.910996 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.910997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:14.911065 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:14.911492 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:14.911496 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:14.912649 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:14.914069 (+ 1420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:14.914070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:14.914071 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:14.914227 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:14.914385 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:14.914386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:14.914386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:14.914443 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:14.914651 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:14.914652 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:14.914696 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 20:17:14.914755 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 20:17:14.914849 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 20:17:14.915412 28054 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:17:14.915705 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.915624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:41403 (local address 127.0.0.1:33506)
0807 20:17:14.915662 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.915687 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 20:17:14.915992 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.915910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:41403 (local address 127.0.0.1:33520)
0807 20:17:14.915947 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.915969 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 20:17:14.916069 28054 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.252:41403: Network error: Client connection negotiation failed: client connection to 127.23.200.252:41403: connect: Connection refused (error 111)
I20260807 20:17:14.947199 28054 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } attempt: 1
I20260807 20:17:14.947687 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.947522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:41403 (local address 127.0.0.1:33530)
0807 20:17:14.947594 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.947646 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 20:17:14.948153 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.948047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:41403 (local address 127.0.0.1:33532)
0807 20:17:14.948091 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.948125 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 20:17:14.948251 28054 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.252:41403: Network error: Client connection negotiation failed: client connection to 127.23.200.252:41403: connect: Connection refused (error 111)
W20260807 20:17:14.952286 28180 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 20:17:14.952486 28180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:17:14.952502 28180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:17:14.952522 28180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:17:14.953703 28180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:17:14.953742 28180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:17:14.953754 28180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:17:14.953765 28180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:17:14.953791 28180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:17:14.955250 28180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:44201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:37141,127.23.200.253:34859,127.23.200.252:41403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.252.keytab
--principal=kudu/127.23.200.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.252:41403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.200.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:17:14.955502 28180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:17:14.955715 28180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:17:14.960584 28190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:14.960677 28180 server_base.cc:1034] running on GCE node
W20260807 20:17:14.960577 28187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:14.960578 28188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:14.961012 28180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:17:14.961328 28180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:17:14.962483 28180 hybrid_clock.cc:648] HybridClock initialized: now 1786133834962467 us; error 29 us; skew 500 ppm
Aug 07 20:17:14 dist-test-slave-2j7r krb5kdc[27916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:17:14.964980 28180 init.cc:377] Logged in from keytab as kudu/127.23.200.252@KRBTEST.COM (short username kudu)
I20260807 20:17:14.965425 28180 webserver.cc:460] Webserver started at http://127.23.200.252:35213/ using document root <none> and password file <none>
I20260807 20:17:14.965622 28180 fs_manager.cc:359] Metadata directory not provided
I20260807 20:17:14.965662 28180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:17:14.965744 28180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:17:14.966784 28180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/data/instance:
uuid: "11af0f481ab643eb8818a971978479b9"
format_stamp: "Formatted at 2026-08-07 20:17:14 on dist-test-slave-2j7r"
server_key: "3d13c5f285719a702ecec1349d4bed26"
server_key_iv: "e239d20a23e738783b3c88d38d17fc00"
server_key_version: "encryptionkey@0"
I20260807 20:17:14.967127 28180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/wal/instance:
uuid: "11af0f481ab643eb8818a971978479b9"
format_stamp: "Formatted at 2026-08-07 20:17:14 on dist-test-slave-2j7r"
server_key: "3d13c5f285719a702ecec1349d4bed26"
server_key_iv: "e239d20a23e738783b3c88d38d17fc00"
server_key_version: "encryptionkey@0"
I20260807 20:17:14.968276 28180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:17:14.968881 28199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:14.969018 28180 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:14.969106 28180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/wal
uuid: "11af0f481ab643eb8818a971978479b9"
format_stamp: "Formatted at 2026-08-07 20:17:14 on dist-test-slave-2j7r"
server_key: "3d13c5f285719a702ecec1349d4bed26"
server_key_iv: "e239d20a23e738783b3c88d38d17fc00"
server_key_version: "encryptionkey@0"
I20260807 20:17:14.969173 28180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:17:14.976197 28172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } attempt: 2
I20260807 20:17:14.976613 28175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.976480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:41403 (local address 127.0.0.1:33548)
0807 20:17:14.976536 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.976577 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 20:17:14.976924 28175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.976854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:41403 (local address 127.0.0.1:33554)
0807 20:17:14.976884 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.976907 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260807 20:17:14.977010 28172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.252:41403: Network error: Client connection negotiation failed: client connection to 127.23.200.252:41403: connect: Connection refused (error 111)
I20260807 20:17:14.985215 28180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:17:14.986958 28180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:17:14.987083 28180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:17:14.991007 28180 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.252:41403
I20260807 20:17:14.991112 28292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.252:41403 every 8 connection(s)
I20260807 20:17:14.991379 28180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/data/info.pb
I20260807 20:17:14.992242 28294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:17:14.993868 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 28180
I20260807 20:17:14.993971 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133562076833-24355-0/minicluster-data/master-2/wal/instance
I20260807 20:17:14.993832 28294 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:17:14.994170 24355 external_mini_cluster.cc:1428] Setting key 1739efd8af5bb05a04e4eb1eb761c70c
Aug 07 20:17:14 dist-test-slave-2j7r krb5kdc[27916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.252@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
Aug 07 20:17:14 dist-test-slave-2j7r krb5kdc[27916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.252@KRBTEST.COM
I20260807 20:17:15.002902 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.994564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37052 (local address 127.23.200.254:37141)
0807 20:17:14.994614 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:14.994616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:14.994864 (+ 248us) server_negotiation.cc:407] Connection header received
0807 20:17:14.995363 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:14.995364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:14.995374 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:14.995448 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:14.995961 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.996434 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.997510 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.997811 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.997883 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:15.000176 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:15.000178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:15.000178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:15.000186 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:15.001592 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.001983 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.001984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.001985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.002045 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.002252 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.002253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.002253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.002372 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:15.002428 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:15.002716 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 20:17:15.002804 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:17:15.003047 28301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.994555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:37141 (local address 127.0.0.1:37052)
0807 20:17:14.994739 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:14.994778 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:14.995267 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:14.995469 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:14.995472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:14.995724 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:14.995879 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:14.995883 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.996506 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.996507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:14.997437 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:14.997439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.997891 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.997892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:14.998055 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:14.998601 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:14.998610 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:15.000090 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:15.001664 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:15.001670 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:15.001673 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:15.001910 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:15.002098 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:15.002098 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:15.002099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:15.002197 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:15.002427 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:15.002434 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:15.002561 (+ 127us) client_negotiation.cc:769] Sending connection context
0807 20:17:15.002675 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 20:17:15.002833 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":25,"thread_start_us":95,"threads_started":1}
I20260807 20:17:15.003427 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:14.994559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33560 (local address 127.23.200.252:41403)
0807 20:17:14.994751 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:14.994761 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:14.994792 (+ 31us) server_negotiation.cc:407] Connection header received
0807 20:17:14.995287 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:14.995299 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:14.995327 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:14.995417 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:14.996014 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.997032 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.997920 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:14.998324 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:14.998376 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:15.000644 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:15.000651 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:15.000651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:15.000657 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:15.002202 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.002515 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.002518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.002519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.002567 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.002751 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.002752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.002752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.002927 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:15.003172 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:15.003252 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 20:17:15.003328 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":84,"threads_started":1}
I20260807 20:17:15.003688 28294 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:17:15 dist-test-slave-2j7r krb5kdc[27916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.252@KRBTEST.COM for kudu/127.23.200.253@KRBTEST.COM
Aug 07 20:17:15 dist-test-slave-2j7r krb5kdc[27916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:17:15.011087 28301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:15.003953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:34859 (local address 127.0.0.1:51012)
0807 20:17:15.003996 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:15.004008 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:15.004043 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:15.004286 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:15.004287 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:15.004412 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:15.004466 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:15.004467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.005299 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.005300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:15.005891 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:15.005893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.006351 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.006352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:15.006420 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:15.006788 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:15.006791 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:15.007978 (+ 1187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:15.010169 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:15.010170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:15.010170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:15.010322 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:15.010510 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:15.010511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:15.010511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:15.010563 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:15.010777 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:15.010778 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:15.010820 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 20:17:15.010884 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 20:17:15.010979 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 20:17:15.011091 28177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:15.004012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51012 (local address 127.23.200.253:34859)
0807 20:17:15.004056 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.004057 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.004065 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:17:15.004090 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.004091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.004101 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.004159 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.004545 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.005234 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.005980 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.006296 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.006355 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:15.008063 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:15.008064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:15.008064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:15.008073 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:15.010087 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.010397 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.010398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.010399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.010451 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.010617 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.010617 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.010618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.010720 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:15.010778 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:15.010915 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 20:17:15.011000 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:17:15.011528 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:15.004290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37060 (local address 127.23.200.254:37141)
0807 20:17:15.004337 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.004339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.004345 (+ 6us) server_negotiation.cc:407] Connection header received
0807 20:17:15.004364 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.004365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.004372 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.004416 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.004943 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.005558 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.006334 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.006606 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.006658 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:15.008675 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:15.008676 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:15.008676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:15.008682 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:15.010064 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.010373 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.010375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.010375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.010433 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.010607 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.010608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.010608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.010718 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:15.011324 (+ 606us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:15.011364 (+ 40us) server_negotiation.cc:299] Negotiation successful
0807 20:17:15.011442 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:17:15.011951 28294 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:17:15 dist-test-slave-2j7r krb5kdc[27916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.252@KRBTEST.COM for kudu/127.23.200.252@KRBTEST.COM
Aug 07 20:17:15 dist-test-slave-2j7r krb5kdc[27916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.253@KRBTEST.COM
I20260807 20:17:15.017997 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:15.012235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33562 (local address 127.23.200.252:41403)
0807 20:17:15.012268 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.012270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.012277 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:17:15.012336 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.012337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.012348 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.012390 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.012605 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.013066 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.013703 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.013933 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.013987 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:15.015640 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:15.015642 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:15.015642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:15.015649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:15.016907 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.017270 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.017271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.017272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.017324 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.017488 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.017489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.017489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.017619 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:15.017680 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:15.017801 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 20:17:15.017921 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":19}
I20260807 20:17:15.018045 28301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:15.012205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:41403 (local address 127.0.0.1:33562)
0807 20:17:15.012246 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:15.012255 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:15.012288 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:15.012391 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:15.012392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:15.012520 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:15.012547 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:15.012548 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.013166 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.013167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:15.013638 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:15.013640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.013990 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.013990 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:15.014035 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:15.014387 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:15.014392 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:15.015551 (+ 1159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:15.016977 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:15.016978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:15.016979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:15.017207 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:15.017389 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:15.017389 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:15.017390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:15.017441 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:15.017677 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:15.017678 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:15.017711 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 20:17:15.017761 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 20:17:15.017862 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 20:17:15.018817 28177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:15.012200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51026 (local address 127.23.200.253:34859)
0807 20:17:15.012236 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.012237 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.012295 (+ 58us) server_negotiation.cc:407] Connection header received
0807 20:17:15.012350 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.012351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.012360 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.012403 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.012815 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.013325 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.013957 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.014152 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.014206 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:15.016210 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:15.016211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:15.016212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:15.016218 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:15.017698 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.017978 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.017979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.017980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.018034 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.018290 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.018292 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.018293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.018429 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:15.018619 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:15.018656 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 20:17:15.018732 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:17:15.019703 28294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9: Bootstrap starting.
I20260807 20:17:15.020217 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:17:15.020570 28294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9: Neither blocks nor log segments found. Creating new log.
I20260807 20:17:15.020792 28294 log.cc:824] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9: Log is configured to *not* fsync() on all Append() calls
I20260807 20:17:15.021587 28294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9: No bootstrap required, opened a new log
I20260807 20:17:15.021134 28010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "09d944951d9c455f8c5cd2f42fcabe08"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37060
I20260807 20:17:15.023007 28294 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } }
I20260807 20:17:15.023133 28294 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11af0f481ab643eb8818a971978479b9, State: Initialized, Role: FOLLOWER
I20260807 20:17:15.023211 28294 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } }
I20260807 20:17:15.023342 28054 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } attempt: 2
I20260807 20:17:15.023535 28294 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:17:15.023501 28304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } } }
I20260807 20:17:15.023595 28304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 20:17:15.025611 28325 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:17:15.025686 28325 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 07 20:17:15 dist-test-slave-2j7r krb5kdc[27916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for kudu/127.23.200.252@KRBTEST.COM
I20260807 20:17:15.025611 28180 master_runner.cc:186] Error getting master registration for 127.23.200.254:37141: OK, instance_id { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" instance_seqno: 1786133834683323 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 20:17:15.028501 28180 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 20:17:15.031183 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:15.023600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:41403 (local address 127.0.0.1:33578)
0807 20:17:15.023656 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:15.023668 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:15.023711 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:15.023841 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:15.023843 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:15.024026 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:15.024076 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:15.024078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.024977 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.024978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:15.025766 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:15.025769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.026236 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.026237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:15.026294 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:15.026757 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:15.026762 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:15.028238 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:15.030103 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:15.030104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:15.030105 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:15.030329 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:15.030528 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:15.030528 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:15.030528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:15.030584 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:15.030870 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:15.030871 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:15.030924 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 20:17:15.030985 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 20:17:15.031082 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 20:17:15.031183 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:15.023650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33578 (local address 127.23.200.252:41403)
0807 20:17:15.023687 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.023689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.023697 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:17:15.023755 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.023756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.023766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.023817 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.024163 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.024887 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.025850 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.026155 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.026227 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:15.028327 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:15.028329 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:15.028329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:15.028337 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:15.030007 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.030394 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.030396 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.030396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.030464 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.030638 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.030639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.030639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.030803 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:15.030879 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:15.031018 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 20:17:15.031098 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:17:15.032718 28054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08: Bootstrap starting.
I20260807 20:17:15.033389 28054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08: Neither blocks nor log segments found. Creating new log.
I20260807 20:17:15.033648 28054 log.cc:824] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08: Log is configured to *not* fsync() on all Append() calls
I20260807 20:17:15.034355 28054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08: No bootstrap required, opened a new log
I20260807 20:17:15.035331 28054 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } }
I20260807 20:17:15.035441 28054 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d944951d9c455f8c5cd2f42fcabe08, State: Initialized, Role: FOLLOWER
I20260807 20:17:15.035513 28054 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } }
I20260807 20:17:15.035754 28331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } } }
I20260807 20:17:15.035802 28054 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:17:15.035842 28331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:17:15.035954 28010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 20:17:15.036034 28010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:17:15.036813 28010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } }
W20260807 20:17:15.037894 28343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:17:15.037955 28343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:17:15.038048 28237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09d944951d9c455f8c5cd2f42fcabe08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "11af0f481ab643eb8818a971978479b9"
I20260807 20:17:15.038173 28237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:17:15.039045 28237 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d944951d9c455f8c5cd2f42fcabe08 in term 1.
I20260807 20:17:15.039336 27973 leader_election.cc:302] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d944951d9c455f8c5cd2f42fcabe08, 11af0f481ab643eb8818a971978479b9; no voters:
I20260807 20:17:15.039475 28331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:17:15.039618 28331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [term 1 LEADER]: Becoming Leader. State: Replica: 09d944951d9c455f8c5cd2f42fcabe08, State: Running, Role: LEADER
I20260807 20:17:15.037075 28010 leader_election.cc:288] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [CANDIDATE]: Term 1 election: Requested vote from peers a6f0e6bf2fe1430190fb9364ce2030f9 (127.23.200.253:34859), 11af0f481ab643eb8818a971978479b9 (127.23.200.252:41403)
I20260807 20:17:15.039721 28331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } }
I20260807 20:17:15.040132 28331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09d944951d9c455f8c5cd2f42fcabe08. Latest consensus state: current_term: 1 leader_uuid: "09d944951d9c455f8c5cd2f42fcabe08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } } }
I20260807 20:17:15.040202 28331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:15.040057 27943 master_runner.cc:186] Error getting master registration for 127.23.200.253:34859: OK, instance_id { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" instance_seqno: 1786133834834946 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 20:17:15.042359 27943 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 20:17:15.043947 28350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:17:15.044613 28350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:17:15.046397 28237 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9 [term 1 FOLLOWER]: Refusing update from remote peer 09d944951d9c455f8c5cd2f42fcabe08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:17:15.046790 28331 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:17:15.048079 28304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09d944951d9c455f8c5cd2f42fcabe08. Latest consensus state: current_term: 1 leader_uuid: "09d944951d9c455f8c5cd2f42fcabe08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } } }
I20260807 20:17:15.048164 28304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:17:15.049791 28331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09d944951d9c455f8c5cd2f42fcabe08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } } }
I20260807 20:17:15.049918 28331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:15.050384 28304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09d944951d9c455f8c5cd2f42fcabe08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } } }
I20260807 20:17:15.050486 28304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:17:15.050841 28365 mvcc.cc:205] Tried to move back new op lower bound from 7316004188347256832 to 7316004188323373056. Current Snapshot: MvccSnapshot[applied={T|T < 7316004188347256832}]
I20260807 20:17:15.051700 28366 mvcc.cc:205] Tried to move back new op lower bound from 7316004188347256832 to 7316004188323373056. Current Snapshot: MvccSnapshot[applied={T|T < 7316004188347256832}]
I20260807 20:17:15.052557 28350 catalog_manager.cc:1347] Generated new cluster ID: 574bf419136e4e9f845dc828e7487ae6
I20260807 20:17:15.052603 28350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:17:15.061504 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:15.042093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37076 (local address 127.23.200.254:37141)
0807 20:17:15.042128 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.042130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.045250 (+ 3120us) server_negotiation.cc:407] Connection header received
0807 20:17:15.045343 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.045344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.045355 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.045383 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.053170 (+ 7787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.053896 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.054884 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.055171 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.055205 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:15.056550 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:15.056552 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:15.056552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:15.056559 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:15.058939 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.060412 (+ 1473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.060414 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.060426 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.060486 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.060779 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.060780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.060780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.060933 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:15.061006 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:15.061320 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 20:17:15.061403 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:17:15.063607 28177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:15.041470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51028 (local address 127.23.200.253:34859)
0807 20:17:15.041522 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.041523 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.042610 (+ 1087us) server_negotiation.cc:407] Connection header received
0807 20:17:15.042668 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.042669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.042693 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.042752 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.046165 (+ 3413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.046849 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.050580 (+ 3731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.050920 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.050962 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:15.056640 (+ 5678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:15.056654 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:15.056654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:15.056663 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:15.059831 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.062590 (+ 2759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.062592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.062592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.062652 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.062891 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.062892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.062892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.063049 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:15.063126 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:15.063416 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 20:17:15.063512 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:17:15.068140 28172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } attempt: 3
I20260807 20:17:15.069999 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:15.041226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33580 (local address 127.23.200.252:41403)
0807 20:17:15.041278 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.041280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.042744 (+ 1464us) server_negotiation.cc:407] Connection header received
0807 20:17:15.042826 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.042827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.042837 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.042864 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.043704 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.044449 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.048265 (+ 3816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.048556 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.048599 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:15.054020 (+ 5421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:15.054022 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:15.054022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:15.054031 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:15.055788 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.069166 (+ 13378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.069168 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.069168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.069244 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.069438 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.069438 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.069439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.069558 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:15.069648 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:15.069816 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 20:17:15.069894 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
Aug 07 20:17:15 dist-test-slave-2j7r krb5kdc[27916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133834, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.253@KRBTEST.COM for kudu/127.23.200.252@KRBTEST.COM
I20260807 20:17:15.080271 28175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:15.068423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:41403 (local address 127.0.0.1:33596)
0807 20:17:15.068485 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:15.068497 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:15.068544 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:15.069008 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:15.069009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:15.069213 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:15.069260 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:15.069262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.071314 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.071315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:15.072148 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:15.072152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.073018 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.073019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:15.073111 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:15.073656 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:15.073662 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:15.075262 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:15.078811 (+ 3549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:15.078812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:15.078813 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:15.079041 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:15.079434 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:15.079435 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:15.079435 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:15.079501 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:15.079963 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:15.079964 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:15.080013 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 20:17:15.080058 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 20:17:15.080164 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 20:17:15.082500 28373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:15.068536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33596 (local address 127.23.200.252:41403)
0807 20:17:15.068770 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.068772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.068780 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:17:15.068832 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.068833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.068844 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.069344 (+ 500us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.069415 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.071159 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.072301 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:15.072899 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:15.075367 (+ 2468us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:15.075495 (+ 128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:15.075496 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:15.075496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:15.075501 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:15.078691 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.079186 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.079187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.079188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.079313 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:15.079582 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:15.079583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:15.079583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:15.079846 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:15.081975 (+ 2129us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:15.082096 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 20:17:15.082253 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":110,"threads_started":1}
I20260807 20:17:15.084446 28172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9: Bootstrap starting.
I20260807 20:17:15.085381 28172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9: Neither blocks nor log segments found. Creating new log.
I20260807 20:17:15.085704 28172 log.cc:824] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9: Log is configured to *not* fsync() on all Append() calls
I20260807 20:17:15.086874 28172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9: No bootstrap required, opened a new log
I20260807 20:17:15.088822 28172 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } }
I20260807 20:17:15.088954 28172 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f0e6bf2fe1430190fb9364ce2030f9, State: Initialized, Role: FOLLOWER
I20260807 20:17:15.089028 28172 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } }
I20260807 20:17:15.089412 28172 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:17:15.090638 28132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "09d944951d9c455f8c5cd2f42fcabe08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9"
I20260807 20:17:15.090749 28132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:17:15.091549 28132 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d944951d9c455f8c5cd2f42fcabe08 in term 1.
I20260807 20:17:15.091748 28130 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9 [term 1 FOLLOWER]: Refusing update from remote peer 09d944951d9c455f8c5cd2f42fcabe08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:17:15.092175 28348 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 20:17:15.094552 28391 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:17:15.094655 28391 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:17:15.094982 28376 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09d944951d9c455f8c5cd2f42fcabe08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } } }
I20260807 20:17:15.095112 28376 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:17:15.095197 28376 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09d944951d9c455f8c5cd2f42fcabe08. Latest consensus state: current_term: 1 leader_uuid: "09d944951d9c455f8c5cd2f42fcabe08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } } }
I20260807 20:17:15.095296 28376 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:17:15.095362 28376 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09d944951d9c455f8c5cd2f42fcabe08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } } }
I20260807 20:17:15.095466 28376 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:17:15.097460 28394 mvcc.cc:205] Tried to move back new op lower bound from 7316004188347256832 to 7316004188323373056. Current Snapshot: MvccSnapshot[applied={T|T < 7316004188347256832}]
I20260807 20:17:15.119705 28348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09d944951d9c455f8c5cd2f42fcabe08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09d944951d9c455f8c5cd2f42fcabe08" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 37141 } } peers { permanent_uuid: "a6f0e6bf2fe1430190fb9364ce2030f9" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 34859 } } peers { permanent_uuid: "11af0f481ab643eb8818a971978479b9" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 41403 } } }
I20260807 20:17:15.119835 28348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08 [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:15.122299 28350 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:17:15.123023 28350 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:17:15.134065 28350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09d944951d9c455f8c5cd2f42fcabe08: Generated new TSK 0
I20260807 20:17:15.134294 28350 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 20:17:15.166877 28177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:15.164890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51042 (local address 127.23.200.253:34859)
0807 20:17:15.164941 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.164944 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.166094 (+ 1150us) server_negotiation.cc:407] Connection header received
0807 20:17:15.166211 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.166213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.166225 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.166255 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.166802 (+ 547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51042: BlockingRecv error: recv got EOF from 127.0.0.1:51042 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 20:17:15.167798 28373 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:15.165351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33600 (local address 127.23.200.252:41403)
0807 20:17:15.165395 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.165396 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.167220 (+ 1824us) server_negotiation.cc:407] Connection header received
0807 20:17:15.167313 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.167314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.167326 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.167354 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.167741 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33600: BlockingRecv error: recv got EOF from 127.0.0.1:33600 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 20:17:15.168522 28060 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:15.165710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37092 (local address 127.23.200.254:37141)
0807 20:17:15.165754 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.165755 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.168019 (+ 2264us) server_negotiation.cc:407] Connection header received
0807 20:17:15.168114 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.168115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.168126 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.168154 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.168457 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37092: BlockingRecv error: recv got EOF from 127.0.0.1:37092 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 20:17:15.170606 28177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:15.169766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51058 (local address 127.23.200.253:34859)
0807 20:17:15.169800 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.169802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.170054 (+ 252us) server_negotiation.cc:407] Connection header received
0807 20:17:15.170194 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.170195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.170215 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.170243 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.170564 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51058: BlockingRecv error: recv got EOF from 127.0.0.1:51058 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 20:17:15.173666 28177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:15.172790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51064 (local address 127.23.200.253:34859)
0807 20:17:15.172824 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.172825 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.173146 (+ 321us) server_negotiation.cc:407] Connection header received
0807 20:17:15.173245 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.173246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.173256 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.173284 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.173624 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51064: BlockingRecv error: recv got EOF from 127.0.0.1:51064 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 20:17:15.177537 28177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:15.176675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51074 (local address 127.23.200.253:34859)
0807 20:17:15.176710 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.176712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.176959 (+ 247us) server_negotiation.cc:407] Connection header received
0807 20:17:15.177048 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.177049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.177059 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.177167 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.177496 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51074: BlockingRecv error: recv got EOF from 127.0.0.1:51074 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 20:17:15.183382 28177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:15.182584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51088 (local address 127.23.200.253:34859)
0807 20:17:15.182623 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.182625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.182875 (+ 250us) server_negotiation.cc:407] Connection header received
0807 20:17:15.182968 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.182969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.182980 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.183007 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.183333 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51088: BlockingRecv error: recv got EOF from 127.0.0.1:51088 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 20:17:15.193557 28177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:15.192631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51104 (local address 127.23.200.253:34859)
0807 20:17:15.192675 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.192677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.192955 (+ 278us) server_negotiation.cc:407] Connection header received
0807 20:17:15.193052 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.193053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.193064 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.193125 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.193504 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51104: BlockingRecv error: recv got EOF from 127.0.0.1:51104 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 20:17:15.211433 28177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:15.210767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51108 (local address 127.23.200.253:34859)
0807 20:17:15.210816 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.210819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.210927 (+ 108us) server_negotiation.cc:407] Connection header received
0807 20:17:15.211007 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.211007 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.211019 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.211066 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.211383 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51108: BlockingRecv error: recv got EOF from 127.0.0.1:51108 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 20:17:15.245383 28177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:15.244708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51122 (local address 127.23.200.253:34859)
0807 20:17:15.244770 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.244772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.244824 (+ 52us) server_negotiation.cc:407] Connection header received
0807 20:17:15.244876 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.244877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.244890 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.244973 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.245318 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51122: BlockingRecv error: recv got EOF from 127.0.0.1:51122 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260807 20:17:15.311178 28177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:15.310498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51126 (local address 127.23.200.253:34859)
0807 20:17:15.310613 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.310615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.310626 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:17:15.310703 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.310704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.310716 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.310765 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.311119 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51126: BlockingRecv error: recv got EOF from 127.0.0.1:51126 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260807 20:17:15.441033 28177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:15.440384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51138 (local address 127.23.200.253:34859)
0807 20:17:15.440467 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.440469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.440549 (+ 80us) server_negotiation.cc:407] Connection header received
0807 20:17:15.440617 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.440618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.440629 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.440677 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.440966 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51138: BlockingRecv error: recv got EOF from 127.0.0.1:51138 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 20:17:15.698958 28177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:15.698248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51152 (local address 127.23.200.253:34859)
0807 20:17:15.698306 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:15.698308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:15.698376 (+ 68us) server_negotiation.cc:407] Connection header received
0807 20:17:15.698447 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:15.698447 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:15.698458 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:15.698509 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:15.698889 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51152: BlockingRecv error: recv got EOF from 127.0.0.1:51152 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:17:16.026147 28325 catalog_manager.cc:1259] Loaded cluster ID: 574bf419136e4e9f845dc828e7487ae6
I20260807 20:17:16.026223 28325 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9: loading cluster ID for follower catalog manager: success
I20260807 20:17:16.026997 28325 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9: acquiring CA information for follower catalog manager: success
I20260807 20:17:16.027261 28325 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 11af0f481ab643eb8818a971978479b9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 20:17:16.095140 28391 catalog_manager.cc:1259] Loaded cluster ID: 574bf419136e4e9f845dc828e7487ae6
I20260807 20:17:16.095216 28391 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9: loading cluster ID for follower catalog manager: success
I20260807 20:17:16.095994 28391 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9: acquiring CA information for follower catalog manager: success
I20260807 20:17:16.096269 28391 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a6f0e6bf2fe1430190fb9364ce2030f9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 20:17:16.214597 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:16.212137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51162 (local address 127.23.200.253:34859)
0807 20:17:16.212242 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:16.212244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:16.212285 (+ 41us) server_negotiation.cc:407] Connection header received
0807 20:17:16.212354 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:16.212354 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:16.212409 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:16.212498 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:16.212640 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:16.213193 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:16.213917 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:16.214113 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:16.214163 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:16.214272 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:16.214390 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:16.214449 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:16.214491 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 20:17:16.214508 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":57,"threads_started":1}
I20260807 20:17:16.218694 28511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:16.216361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33608 (local address 127.23.200.252:41403)
0807 20:17:16.216485 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:16.216487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:16.216496 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:17:16.216538 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:16.216539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:16.216580 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:16.216650 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:16.216754 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:16.217292 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:16.217992 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:16.218187 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:16.218241 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:16.218377 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:16.218484 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:16.218548 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:16.218585 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 20:17:16.218606 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":53,"threads_started":1}
I20260807 20:17:16.223193 28511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:16.220386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33610 (local address 127.23.200.252:41403)
0807 20:17:16.220432 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:16.220434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:16.220888 (+ 454us) server_negotiation.cc:407] Connection header received
0807 20:17:16.220939 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:16.220940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:16.220950 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:16.220993 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:16.221119 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:16.221564 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:16.222597 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:16.222777 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:16.222826 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:16.222976 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:16.223007 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:16.223056 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:16.223103 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 20:17:16.223123 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:17:16.223750 28521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:16.220390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37098 (local address 127.23.200.254:37141)
0807 20:17:16.220592 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:16.220594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:16.220601 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:17:16.220646 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:16.220646 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:16.220681 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:16.220756 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:16.220931 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:16.221677 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:16.222856 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:16.223100 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:16.223151 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:16.223397 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:16.223514 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:16.223609 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:16.223643 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 20:17:16.223667 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":60,"threads_started":1}
I20260807 20:17:16.224212 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:16.220495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51170 (local address 127.23.200.253:34859)
0807 20:17:16.220540 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:16.220541 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:16.220600 (+ 59us) server_negotiation.cc:407] Connection header received
0807 20:17:16.220660 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:16.220661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:16.220670 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:16.220714 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:16.220805 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:16.221557 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:16.223444 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:16.223718 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:16.223755 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:16.223973 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:16.224005 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:16.224030 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:16.224129 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 20:17:16.224152 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 20:17:16.225490 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 27943
I20260807 20:17:16.235800 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 28066
I20260807 20:17:16.250797 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 28180
2026-08-07T20:17:16Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2962 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:17:16 dist-test-slave-2j7r krb5kdc[28527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:17:16 dist-test-slave-2j7r krb5kdc[28527](info): set up 2 sockets
Aug 07 20:17:16 dist-test-slave-2j7r krb5kdc[28527](info): commencing operation
krb5kdc: starting...
W20260807 20:17:17.400293 24355 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/test-user.keytab.
Aug 07 20:17:17 dist-test-slave-2j7r krb5kdc[28527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:17:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:17:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.200.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
Entry for principal kudu/127.23.200.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab.
I20260807 20:17:17.521885 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:33539
--webserver_interface=127.23.200.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:36763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:33539,127.23.200.253:33859,127.23.200.252:33073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:17:17.604950 28547 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 20:17:17.605201 28547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:17:17.605217 28547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:17:17.605237 28547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:17:17.606498 28547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:17:17.606542 28547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:17:17.606554 28547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:17:17.606565 28547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:17:17.606576 28547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:17:17.608067 28547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:36763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:33539,127.23.200.253:33859,127.23.200.252:33073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.254.keytab
--principal=kudu/127.23.200.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.254:33539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.200.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:17:17.608317 28547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:17:17.608539 28547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:17:17.613788 28558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:17.613811 28555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:17.613850 28547 server_base.cc:1034] running on GCE node
W20260807 20:17:17.613811 28560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:17.614326 28547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:17:17.614600 28547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:17:17.615739 28547 hybrid_clock.cc:648] HybridClock initialized: now 1786133837615726 us; error 29 us; skew 500 ppm
Aug 07 20:17:17 dist-test-slave-2j7r krb5kdc[28527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:17:17.621256 28547 init.cc:377] Logged in from keytab as kudu/127.23.200.254@KRBTEST.COM (short username kudu)
I20260807 20:17:17.621692 28547 webserver.cc:460] Webserver started at http://127.23.200.254:46175/ using document root <none> and password file <none>
I20260807 20:17:17.621896 28547 fs_manager.cc:359] Metadata directory not provided
I20260807 20:17:17.621954 28547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:17:17.622085 28547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:17:17.623143 28547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/data/instance:
uuid: "f6f97721102b45a98aaa102769b530e5"
format_stamp: "Formatted at 2026-08-07 20:17:17 on dist-test-slave-2j7r"
server_key: "1da40a9c808493481ad419d60159f009"
server_key_iv: "d9ccfd4d64b0fdc87d3fdc025ee4552f"
server_key_version: "encryptionkey@0"
I20260807 20:17:17.623535 28547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/wal/instance:
uuid: "f6f97721102b45a98aaa102769b530e5"
format_stamp: "Formatted at 2026-08-07 20:17:17 on dist-test-slave-2j7r"
server_key: "1da40a9c808493481ad419d60159f009"
server_key_iv: "d9ccfd4d64b0fdc87d3fdc025ee4552f"
server_key_version: "encryptionkey@0"
I20260807 20:17:17.624780 28547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:17:17.625545 28577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:17.625698 28547 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:17.625787 28547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/wal
uuid: "f6f97721102b45a98aaa102769b530e5"
format_stamp: "Formatted at 2026-08-07 20:17:17 on dist-test-slave-2j7r"
server_key: "1da40a9c808493481ad419d60159f009"
server_key_iv: "d9ccfd4d64b0fdc87d3fdc025ee4552f"
server_key_version: "encryptionkey@0"
I20260807 20:17:17.625847 28547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:17:17.641280 28547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:17:17.643213 28547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:17:17.643362 28547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:17:17.647449 28547 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.254:33539
I20260807 20:17:17.647490 28656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.254:33539 every 8 connection(s)
I20260807 20:17:17.647840 28547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/data/info.pb
I20260807 20:17:17.648772 28658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:17:17.650374 28658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:17:17 dist-test-slave-2j7r krb5kdc[28527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:17:17.657172 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 28547
I20260807 20:17:17.657285 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-0/wal/instance
I20260807 20:17:17.657531 24355 external_mini_cluster.cc:1428] Setting key 378e20b6aaaeb96230fe33fc2b73da23
I20260807 20:17:17.659693 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.651059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39506 (local address 127.23.200.254:33539)
0807 20:17:17.651236 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:17.651246 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:17.651316 (+ 70us) server_negotiation.cc:407] Connection header received
0807 20:17:17.651902 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:17.651904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:17.651922 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:17.652008 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:17.652524 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.653364 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.654211 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.654522 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.654603 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.656535 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:17.656543 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:17.656543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:17.656549 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:17.658183 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.658528 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.658532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.658532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.658636 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.658835 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.658836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.658836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.659113 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:17.659197 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:17.659395 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 20:17:17.659504 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":80,"threads_started":1}
I20260807 20:17:17.659701 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.651038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:33539 (local address 127.0.0.1:39506)
0807 20:17:17.651212 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.651246 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:17.651764 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:17.652032 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:17.652035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:17.652296 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:17.652453 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:17.652457 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.653448 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.653449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:17.654137 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:17.654139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.654588 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.654589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:17.654642 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.655099 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:17.655107 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:17.656463 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:17.658257 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:17.658264 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:17.658266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:17.658457 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:17.658698 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:17.658699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:17.658699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:17.658781 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:17.659170 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:17.659173 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:17.659244 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 20:17:17.659340 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 20:17:17.659576 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":79,"thread_start_us":83,"threads_started":1}
I20260807 20:17:17.660712 28658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:17:17.661052 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.660930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:33859 (local address 127.0.0.1:54110)
0807 20:17:17.660977 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.661028 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:33859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 20:17:17.661196 28581 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.200.253:33859: connect: Connection refused (error 111)
I20260807 20:17:17.661630 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.661537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:33859 (local address 127.0.0.1:54120)
0807 20:17:17.661581 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.661610 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:33859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 20:17:17.661729 28658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.253:33859: Network error: Client connection negotiation failed: client connection to 127.23.200.253:33859: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.23.200.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.253.keytab.
Entry for principal kudu/127.23.200.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.253.keytab.
I20260807 20:17:17.684159 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.253:33859
--webserver_interface=127.23.200.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.253.keytab
--principal=kudu/127.23.200.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:36763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:33539,127.23.200.253:33859,127.23.200.252:33073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:17:17.710980 28658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } attempt: 1
I20260807 20:17:17.711588 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.711407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:33859 (local address 127.0.0.1:54122)
0807 20:17:17.711501 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.711550 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:33859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 20:17:17.711992 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.711896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:33859 (local address 127.0.0.1:54136)
0807 20:17:17.711942 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.711968 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:33859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 20:17:17.712095 28658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.253:33859: Network error: Client connection negotiation failed: client connection to 127.23.200.253:33859: connect: Connection refused (error 111)
W20260807 20:17:17.768007 28670 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 20:17:17.768229 28670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:17:17.768254 28670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:17:17.768278 28670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:17:17.769483 28670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:17:17.769526 28670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:17:17.769546 28670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:17:17.769562 28670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:17:17.769578 28670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:17:17.771100 28670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:36763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:33539,127.23.200.253:33859,127.23.200.252:33073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.253.keytab
--principal=kudu/127.23.200.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.253:33859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.200.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:17:17.771348 28670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:17:17.771560 28670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:17:17.776391 28684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:17.776402 28679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:17.776440 28670 server_base.cc:1034] running on GCE node
W20260807 20:17:17.776402 28682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:17.776871 28670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:17:17.777182 28670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:17:17.778365 28670 hybrid_clock.cc:648] HybridClock initialized: now 1786133837778344 us; error 34 us; skew 500 ppm
I20260807 20:17:17.780247 28658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } attempt: 2
I20260807 20:17:17.780638 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.780517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:33859 (local address 127.0.0.1:54142)
0807 20:17:17.780574 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.780609 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:33859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
Aug 07 20:17:17 dist-test-slave-2j7r krb5kdc[28527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:17:17.780974 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.780874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:33859 (local address 127.0.0.1:54146)
0807 20:17:17.780916 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.780945 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.253:33859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 20:17:17.781078 28658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.253:33859: Network error: Client connection negotiation failed: client connection to 127.23.200.253:33859: connect: Connection refused (error 111)
I20260807 20:17:17.781237 28670 init.cc:377] Logged in from keytab as kudu/127.23.200.253@KRBTEST.COM (short username kudu)
I20260807 20:17:17.781653 28670 webserver.cc:460] Webserver started at http://127.23.200.253:38053/ using document root <none> and password file <none>
I20260807 20:17:17.781852 28670 fs_manager.cc:359] Metadata directory not provided
I20260807 20:17:17.781896 28670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:17:17.781991 28670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:17:17.783008 28670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/data/instance:
uuid: "31a6d05f60d5462bb1fb63e5227e8bac"
format_stamp: "Formatted at 2026-08-07 20:17:17 on dist-test-slave-2j7r"
server_key: "97ca815e947f25d43d8f1ff9874c3756"
server_key_iv: "328504fbc6159ab04a936b6e46fcd53b"
server_key_version: "encryptionkey@0"
I20260807 20:17:17.783329 28670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/wal/instance:
uuid: "31a6d05f60d5462bb1fb63e5227e8bac"
format_stamp: "Formatted at 2026-08-07 20:17:17 on dist-test-slave-2j7r"
server_key: "97ca815e947f25d43d8f1ff9874c3756"
server_key_iv: "328504fbc6159ab04a936b6e46fcd53b"
server_key_version: "encryptionkey@0"
I20260807 20:17:17.784619 28670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:17:17.785394 28696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:17.785557 28670 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:17:17.785650 28670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/wal
uuid: "31a6d05f60d5462bb1fb63e5227e8bac"
format_stamp: "Formatted at 2026-08-07 20:17:17 on dist-test-slave-2j7r"
server_key: "97ca815e947f25d43d8f1ff9874c3756"
server_key_iv: "328504fbc6159ab04a936b6e46fcd53b"
server_key_version: "encryptionkey@0"
I20260807 20:17:17.785715 28670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:17:17.805989 28670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:17:17.808234 28670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:17:17.808386 28670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:17:17.812256 28670 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.253:33859
I20260807 20:17:17.812299 28783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.253:33859 every 8 connection(s)
I20260807 20:17:17.812628 28670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/data/info.pb
I20260807 20:17:17.813580 28787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:17:17.815116 28787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:17:17.820329 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 28670
I20260807 20:17:17.820437 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-1/wal/instance
I20260807 20:17:17.820616 24355 external_mini_cluster.cc:1428] Setting key bde0ab74be550ffe17a535d3ad661d7c
Aug 07 20:17:17 dist-test-slave-2j7r krb5kdc[28527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.253@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:17:17.824172 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.815816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39514 (local address 127.23.200.254:33539)
0807 20:17:17.815895 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:17.815898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:17.816075 (+ 177us) server_negotiation.cc:407] Connection header received
0807 20:17:17.816631 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:17.816633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:17.816646 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:17.816696 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:17.817325 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.817819 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.818786 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.819012 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.819070 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.821241 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:17.821243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:17.821243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:17.821251 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:17.822747 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.823147 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.823148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.823148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.823215 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.823493 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.823493 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.823494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.823622 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:17.823676 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:17.824023 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 20:17:17.824102 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 20:17:17.824365 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.815752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:33539 (local address 127.0.0.1:39514)
0807 20:17:17.815948 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.815987 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:17.816522 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:17.816765 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:17.816769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:17.817016 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:17.817233 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:17.817237 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.817935 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.817936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:17.818704 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:17.818705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.819085 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.819086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:17.819207 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.819662 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:17.819671 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:17.821127 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:17.822855 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:17.822862 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:17.822864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:17.823079 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:17.823292 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:17.823293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:17.823293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:17.823435 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:17.823681 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:17.823687 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:17.823842 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 20:17:17.823974 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 20:17:17.824138 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":85,"threads_started":1}
I20260807 20:17:17.825003 28787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.200.252@KRBTEST.COM; defaulting to no policy
Aug 07 20:17:17 dist-test-slave-2j7r krb5kdc[28527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.253@KRBTEST.COM for kudu/127.23.200.253@KRBTEST.COM
I20260807 20:17:17.832423 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.825292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:33859 (local address 127.0.0.1:54156)
0807 20:17:17.825482 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.825491 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:17.825528 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:17.825757 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:17.825758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:17.825866 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:17.825914 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:17.825915 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.827157 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.827158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:17.827668 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:17.827669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.828333 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.828334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:17.828390 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.828757 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:17.828761 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:17.829971 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:17.831515 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:17.831516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:17.831516 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:17.831668 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:17.831825 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:17.831825 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:17.831825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:17.831875 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:17.832151 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:17.832151 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:17.832184 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 20:17:17.832227 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 20:17:17.832303 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 20:17:17.832509 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.825360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54156 (local address 127.23.200.253:33859)
0807 20:17:17.825598 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:17.825603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:17.825629 (+ 26us) server_negotiation.cc:407] Connection header received
0807 20:17:17.825653 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:17.825655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:17.825669 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:17.825750 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:17.825999 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.827063 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.827749 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.828261 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.828345 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.830050 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:17.830059 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:17.830060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:17.830066 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:17.831444 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.831730 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.831733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.831733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.831782 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.831950 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.831951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.831951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.832104 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:17.832165 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:17.832257 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 20:17:17.832349 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":203,"thread_start_us":86,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.200.252@KRBTEST.COM" created.
I20260807 20:17:17.833108 28787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:17:17.833455 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.833336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:33073 (local address 127.0.0.1:57146)
0807 20:17:17.833392 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.833438 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 20:17:17.833577 28701 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.200.252:33073: connect: Connection refused (error 111)
I20260807 20:17:17.833978 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.833884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:33073 (local address 127.0.0.1:57154)
0807 20:17:17.833925 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.833962 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 20:17:17.834074 28787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.252:33073: Network error: Client connection negotiation failed: client connection to 127.23.200.252:33073: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.200.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.252.keytab.
Entry for principal kudu/127.23.200.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.252.keytab.
I20260807 20:17:17.847813 24355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskISIRft/build/release/bin/kudu
/tmp/dist-test-taskISIRft/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.252:33073
--webserver_interface=127.23.200.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.252.keytab
--principal=kudu/127.23.200.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.200.212:36763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.200.254:33539,127.23.200.253:33859,127.23.200.252:33073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:17:17.872234 28658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } attempt: 3
Aug 07 20:17:17 dist-test-slave-2j7r krb5kdc[28527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for kudu/127.23.200.253@KRBTEST.COM
I20260807 20:17:17.879212 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.872707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54170 (local address 127.23.200.253:33859)
0807 20:17:17.872759 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:17.872761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:17.872773 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:17:17.872822 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:17.872823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:17.872835 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:17.872885 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:17.873237 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.873732 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.874495 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.874729 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.874788 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.876859 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:17.876861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:17.876861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:17.876870 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:17.878139 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.878493 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.878494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.878495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.878550 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.878718 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.878718 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.878719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.878832 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:17.878888 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:17.879033 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 20:17:17.879109 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:17:17.879212 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.872638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:33859 (local address 127.0.0.1:54170)
0807 20:17:17.872701 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.872713 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:17.872766 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:17.872896 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:17.872897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:17.873065 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:17.873167 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:17.873170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.873812 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.873813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:17.874402 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:17.874404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.874797 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.874798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:17.874874 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.875330 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:17.875334 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:17.876774 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:17.878219 (+ 1445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:17.878221 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:17.878221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:17.878414 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:17.878606 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:17.878607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:17.878608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:17.878667 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:17.878887 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:17.878888 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:17.878945 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 20:17:17.879003 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 20:17:17.879106 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 20:17:17.879630 28658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:17:17.879956 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.879855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:33073 (local address 127.0.0.1:57168)
0807 20:17:17.879896 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.879924 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 20:17:17.880287 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.880209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:33073 (local address 127.0.0.1:57184)
0807 20:17:17.880250 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.880272 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 20:17:17.880442 28658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.252:33073: Network error: Client connection negotiation failed: client connection to 127.23.200.252:33073: connect: Connection refused (error 111)
I20260807 20:17:17.883270 28787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } attempt: 1
I20260807 20:17:17.883599 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.883500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:33073 (local address 127.0.0.1:57196)
0807 20:17:17.883543 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.883569 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 20:17:17.883857 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.883782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:33073 (local address 127.0.0.1:57206)
0807 20:17:17.883814 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.883835 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260807 20:17:17.883942 28787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.252:33073: Network error: Client connection negotiation failed: client connection to 127.23.200.252:33073: connect: Connection refused (error 111)
I20260807 20:17:17.911567 28658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } attempt: 1
I20260807 20:17:17.912035 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.911894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:33073 (local address 127.0.0.1:57210)
0807 20:17:17.911968 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.912002 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 20:17:17.912505 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.912411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:33073 (local address 127.0.0.1:57218)
0807 20:17:17.912458 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.912485 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 20:17:17.912613 28658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.252:33073: Network error: Client connection negotiation failed: client connection to 127.23.200.252:33073: connect: Connection refused (error 111)
W20260807 20:17:17.933189 28796 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 20:17:17.933416 28796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:17:17.933441 28796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:17:17.933470 28796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:17:17.934664 28796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:17:17.934710 28796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:17:17.934729 28796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:17:17.934746 28796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:17:17.934760 28796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:17:17.936281 28796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.200.212:36763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.200.254:33539,127.23.200.253:33859,127.23.200.252:33073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/krb5kdc/kudu_127.23.200.252.keytab
--principal=kudu/127.23.200.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.200.252:33073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.200.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:17:17.936532 28796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:17:17.936743 28796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:17:17.942152 28804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:17.942169 28796 server_base.cc:1034] running on GCE node
W20260807 20:17:17.942171 28806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:17:17.942138 28803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:17:17.942534 28796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:17:17.942817 28796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:17:17.943979 28796 hybrid_clock.cc:648] HybridClock initialized: now 1786133837943966 us; error 36 us; skew 500 ppm
Aug 07 20:17:17 dist-test-slave-2j7r krb5kdc[28527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:17:17.946564 28796 init.cc:377] Logged in from keytab as kudu/127.23.200.252@KRBTEST.COM (short username kudu)
I20260807 20:17:17.947047 28796 webserver.cc:460] Webserver started at http://127.23.200.252:41097/ using document root <none> and password file <none>
I20260807 20:17:17.947269 28796 fs_manager.cc:359] Metadata directory not provided
I20260807 20:17:17.947312 28796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:17:17.947443 28796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:17:17.948515 28796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/data/instance:
uuid: "c9535956ec4a4e578ce7b159188190e7"
format_stamp: "Formatted at 2026-08-07 20:17:17 on dist-test-slave-2j7r"
server_key: "81cba1da933c0e57249e6769f6c98106"
server_key_iv: "f65d04987485f56a3c0f4a38526ab76f"
server_key_version: "encryptionkey@0"
I20260807 20:17:17.948853 28796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/wal/instance:
uuid: "c9535956ec4a4e578ce7b159188190e7"
format_stamp: "Formatted at 2026-08-07 20:17:17 on dist-test-slave-2j7r"
server_key: "81cba1da933c0e57249e6769f6c98106"
server_key_iv: "f65d04987485f56a3c0f4a38526ab76f"
server_key_version: "encryptionkey@0"
I20260807 20:17:17.950135 28796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 20:17:17.950814 28820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:17:17.950980 28796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:17:17.951054 28796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/data,/tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/wal
uuid: "c9535956ec4a4e578ce7b159188190e7"
format_stamp: "Formatted at 2026-08-07 20:17:17 on dist-test-slave-2j7r"
server_key: "81cba1da933c0e57249e6769f6c98106"
server_key_iv: "f65d04987485f56a3c0f4a38526ab76f"
server_key_version: "encryptionkey@0"
I20260807 20:17:17.951118 28796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:17:17.952072 28787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } attempt: 2
I20260807 20:17:17.952454 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.952338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:33073 (local address 127.0.0.1:57224)
0807 20:17:17.952387 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.952420 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 20:17:17.952785 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.952695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:33073 (local address 127.0.0.1:57236)
0807 20:17:17.952735 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.952758 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.200.252:33073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 20:17:17.952878 28787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.200.252:33073: Network error: Client connection negotiation failed: client connection to 127.23.200.252:33073: connect: Connection refused (error 111)
I20260807 20:17:17.966746 28796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:17:17.968878 28796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:17:17.969012 28796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:17:17.972703 28796 rpc_server.cc:307] RPC server started. Bound to: 127.23.200.252:33073
I20260807 20:17:17.972738 28901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.200.252:33073 every 8 connection(s)
I20260807 20:17:17.973105 28796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/data/info.pb
I20260807 20:17:17.973978 28903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:17:17.975550 28903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:17:17 dist-test-slave-2j7r krb5kdc[28527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.252@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:17:17.983332 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.976222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39524 (local address 127.23.200.254:33539)
0807 20:17:17.976282 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:17.976284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:17.976465 (+ 181us) server_negotiation.cc:407] Connection header received
0807 20:17:17.976907 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:17.976908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:17.976917 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:17.976982 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:17.977585 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.978059 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.978974 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.979149 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.979194 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.981184 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:17.981186 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:17.981186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:17.981193 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:17.982289 (+ 1096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.982641 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.982641 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.982642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.982683 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.982850 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.982851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.982851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.982958 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:17.983004 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:17.983207 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 20:17:17.983267 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:17:17.983371 24355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskISIRft/build/release/bin/kudu as pid 28796
I20260807 20:17:17.983484 24355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskISIRft/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133562076833-24355-0/minicluster-data/master-2/wal/instance
I20260807 20:17:17.983590 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.976185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.254:33539 (local address 127.0.0.1:39524)
0807 20:17:17.976344 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.976385 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:17.976832 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:17.977046 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:17.977050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:17.977320 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:17.977514 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:17.977518 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.978129 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.978130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:17.978910 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:17.978913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.979213 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.979214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:17.979318 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.979758 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:17.979767 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:17.981053 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:17.982360 (+ 1307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:17.982366 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:17.982368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:17.982588 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:17.982725 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:17.982726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:17.982726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:17.982811 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:17.982999 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:17.983002 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:17.983109 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 20:17:17.983185 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 20:17:17.983336 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260807 20:17:17.983708 24355 external_mini_cluster.cc:1428] Setting key abe18bf0b916247d0eb44d43dce3ab2c
I20260807 20:17:17.984114 28903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:17:17.987749 28658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } attempt: 2
Aug 07 20:17:17 dist-test-slave-2j7r krb5kdc[28527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.252@KRBTEST.COM for kudu/127.23.200.253@KRBTEST.COM
Aug 07 20:17:17 dist-test-slave-2j7r krb5kdc[28527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.252@KRBTEST.COM
I20260807 20:17:17.990950 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.984370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.253:33859 (local address 127.0.0.1:54180)
0807 20:17:17.984401 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.984417 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:17.984445 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:17.984643 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:17.984643 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:17.984751 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:17.984794 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:17.984795 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.985550 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.985551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:17.986313 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:17.986315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.986748 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.986748 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:17.986793 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.987159 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:17.987163 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:17.988551 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:17.990033 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:17.990033 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:17.990034 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:17.990219 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:17.990394 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:17.990395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:17.990395 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:17.990458 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:17.990664 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:17.990665 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:17.990708 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 20:17:17.990773 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 20:17:17.990857 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":1}
I20260807 20:17:17.990985 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.984469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54180 (local address 127.23.200.253:33859)
0807 20:17:17.984532 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:17.984534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:17.984542 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:17:17.984566 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:17.984568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:17.984577 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:17.984629 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:17.984857 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.985484 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.986378 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.986684 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.986754 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.988635 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:17.988636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:17.988636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:17.988644 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:17.989963 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.990281 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.990282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.990283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.990340 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.990505 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.990506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.990506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.990611 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:17.990665 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:17.990810 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 20:17:17.990884 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:17:17.991355 28903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:17:17 dist-test-slave-2j7r krb5kdc[28527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.254@KRBTEST.COM for kudu/127.23.200.252@KRBTEST.COM
I20260807 20:17:17.994079 28907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.984078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57242 (local address 127.23.200.252:33073)
0807 20:17:17.984268 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:17.984273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:17.984299 (+ 26us) server_negotiation.cc:407] Connection header received
0807 20:17:17.984375 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:17.984378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:17.984393 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:17.984555 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:17.985257 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.986208 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.987136 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.987628 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.987688 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.990565 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:17.990574 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:17.990574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:17.990580 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:17.992342 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.992675 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.992679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.992679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.992735 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.993210 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.993212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.993212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.993390 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:17.993535 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:17.993865 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 20:17:17.993980 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":78,"threads_started":1}
Aug 07 20:17:17 dist-test-slave-2j7r krb5kdc[28527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.252@KRBTEST.COM for kudu/127.23.200.252@KRBTEST.COM
I20260807 20:17:17.995782 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.988001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:33073 (local address 127.0.0.1:57258)
0807 20:17:17.988174 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.988184 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:17.988224 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:17.988419 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:17.988420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:17.988559 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:17.988595 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:17.988597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.989488 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.989489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:17.990247 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:17.990251 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.990726 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.990727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:17.990789 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.991366 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:17.991370 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:17.992721 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:17.994733 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:17.994734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:17.994734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:17.994942 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:17.995169 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:17.995170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:17.995171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:17.995238 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:17.995492 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:17.995492 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:17.995536 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 20:17:17.995601 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 20:17:17.995689 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260807 20:17:17.996080 28908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.988064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57258 (local address 127.23.200.252:33073)
0807 20:17:17.988303 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:17.988305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:17.988311 (+ 6us) server_negotiation.cc:407] Connection header received
0807 20:17:17.988328 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:17.988328 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:17.988338 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:17.988400 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:17.988676 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.989406 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.990342 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.990629 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.990718 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.992826 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:17.992828 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:17.992828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:17.992833 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:17.994637 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.995018 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.995019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.995020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.995079 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.995302 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.995303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.995303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.995424 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:17.995494 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:17.995891 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 20:17:17.995961 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":77,"threads_started":1}
I20260807 20:17:17.997979 28658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5: Bootstrap starting.
I20260807 20:17:17.998415 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.991557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:33073 (local address 127.0.0.1:57268)
0807 20:17:17.991605 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:17.991616 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:17.991657 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:17.991875 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:17.991876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:17.992007 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:17.992038 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:17.992039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.993225 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.993226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:17.993855 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:17.993858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.994275 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.994276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:17.994331 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.994625 (+ 294us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:17.994629 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:17.995817 (+ 1188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:17.997482 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:17.997483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:17.997484 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:17.997627 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:17.997781 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:17.997782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:17.997782 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:17.997832 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:17.998086 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:17.998087 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:17.998132 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 20:17:17.998204 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 20:17:17.998299 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 20:17:17.998469 28913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.991626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57268 (local address 127.23.200.252:33073)
0807 20:17:17.991761 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:17.991763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:17.991769 (+ 6us) server_negotiation.cc:407] Connection header received
0807 20:17:17.991784 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:17.991785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:17.991794 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:17.991859 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:17.992115 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.993137 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.993948 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.994206 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.994267 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:17.995889 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:17.995890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:17.995891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:17.995896 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:17.997423 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.997681 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.997681 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.997682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.997731 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:17.997879 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:17.997879 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:17.997880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:17.998000 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:17.998082 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:17.998232 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 20:17:17.998343 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260807 20:17:17.998786 28658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5: Neither blocks nor log segments found. Creating new log.
I20260807 20:17:17.999092 28658 log.cc:824] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5: Log is configured to *not* fsync() on all Append() calls
I20260807 20:17:17.999560 28903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7: Bootstrap starting.
Aug 07 20:17:17 dist-test-slave-2j7r krb5kdc[28527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.254@KRBTEST.COM
I20260807 20:17:18.000118 28903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7: Neither blocks nor log segments found. Creating new log.
I20260807 20:17:18.000280 28658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5: No bootstrap required, opened a new log
I20260807 20:17:18.000308 28903 log.cc:824] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7: Log is configured to *not* fsync() on all Append() calls
I20260807 20:17:18.000954 28903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7: No bootstrap required, opened a new log
I20260807 20:17:18.002277 28658 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } }
I20260807 20:17:18.002380 28658 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6f97721102b45a98aaa102769b530e5, State: Initialized, Role: FOLLOWER
I20260807 20:17:18.002461 28658 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } }
I20260807 20:17:18.002537 28903 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } }
I20260807 20:17:18.002671 28903 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9535956ec4a4e578ce7b159188190e7, State: Initialized, Role: FOLLOWER
I20260807 20:17:18.002714 28919 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } } }
I20260807 20:17:18.002774 28658 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:17:18.002748 28903 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } }
I20260807 20:17:18.002789 28919 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:17:18.002992 28920 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } } }
I20260807 20:17:18.003037 28903 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:17:18.003089 28920 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:17:18.003327 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:17.995054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39532 (local address 127.23.200.254:33539)
0807 20:17:17.995090 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:17.995092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:17.995098 (+ 6us) server_negotiation.cc:407] Connection header received
0807 20:17:17.995115 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:17.995116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:17.995125 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:17.995170 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:17.995743 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.996402 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.997322 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:17.997583 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:17.997647 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.000554 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:18.000555 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:18.000556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:18.000562 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:18.002043 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:18.002416 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:18.002417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:18.002418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:18.002474 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:18.002696 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:18.002697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:18.002697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:18.002827 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:18.002945 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:18.003166 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 20:17:18.003246 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 20:17:18.005560 28954 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:17:18.005610 28954 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260807 20:17:18.006296 28949 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:17:18.006349 28949 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:17:18.006093 28547 master_runner.cc:186] Error getting master registration for 127.23.200.253:33859: OK, instance_id { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" instance_seqno: 1786133837811214 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 20:17:18.008137 28547 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 07 20:17:18 dist-test-slave-2j7r krb5kdc[28527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.200.253@KRBTEST.COM
I20260807 20:17:18.008387 28796 master_runner.cc:186] Error getting master registration for 127.23.200.253:33859: OK, instance_id { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" instance_seqno: 1786133837811214 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 20:17:18.010563 28796 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 20:17:18.012607 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:18.004277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54192 (local address 127.23.200.253:33859)
0807 20:17:18.004319 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.004320 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.004328 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:17:18.004349 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.004350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.004360 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.004409 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.005001 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.005661 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.006511 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.006788 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.006845 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.009380 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:18.009382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:18.009382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:18.009389 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:18.011581 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:18.011878 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:18.011879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:18.011879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:18.011937 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:18.012090 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:18.012090 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:18.012091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:18.012220 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:18.012386 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:18.012434 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 20:17:18.012513 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:17:18.013901 24355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:17:18.014866 28617 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f6f97721102b45a98aaa102769b530e5"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39532
I20260807 20:17:18.015002 28617 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 20:17:18.015054 28617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:17:18.015961 28617 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } }
I20260807 20:17:18.016196 28617 leader_election.cc:288] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [CANDIDATE]: Term 1 election: Requested vote from peers 31a6d05f60d5462bb1fb63e5227e8bac (127.23.200.253:33859), c9535956ec4a4e578ce7b159188190e7 (127.23.200.252:33073)
I20260807 20:17:18.016479 28862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6f97721102b45a98aaa102769b530e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c9535956ec4a4e578ce7b159188190e7"
I20260807 20:17:18.016598 28862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:17:18.017661 28862 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6f97721102b45a98aaa102769b530e5 in term 1.
I20260807 20:17:18.017845 28581 leader_election.cc:302] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9535956ec4a4e578ce7b159188190e7, f6f97721102b45a98aaa102769b530e5; no voters:
I20260807 20:17:18.017992 28919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:17:18.018203 28919 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [term 1 LEADER]: Becoming Leader. State: Replica: f6f97721102b45a98aaa102769b530e5, State: Running, Role: LEADER
I20260807 20:17:18.018296 28919 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } }
I20260807 20:17:18.018720 28919 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6f97721102b45a98aaa102769b530e5. Latest consensus state: current_term: 1 leader_uuid: "f6f97721102b45a98aaa102769b530e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } } }
I20260807 20:17:18.018810 28919 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:18.029006 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:18.019039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54200 (local address 127.23.200.253:33859)
0807 20:17:18.019078 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.019080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.019375 (+ 295us) server_negotiation.cc:407] Connection header received
0807 20:17:18.019508 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.019509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.019532 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.019564 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.020789 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.021575 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.023140 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.023450 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.023496 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.025680 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:18.025682 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:18.025682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:18.025701 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:18.027277 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:18.027869 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:18.027870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:18.027871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:18.027950 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:18.028278 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:18.028279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:18.028280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:18.028418 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:18.028493 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:18.028810 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 20:17:18.028893 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:17:18.029798 28971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:17:18.030037 28971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:17:18.032052 28862 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7 [term 1 FOLLOWER]: Refusing update from remote peer f6f97721102b45a98aaa102769b530e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:17:18.032460 28919 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:17:18.033892 28920 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6f97721102b45a98aaa102769b530e5. Latest consensus state: current_term: 1 leader_uuid: "f6f97721102b45a98aaa102769b530e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } } }
I20260807 20:17:18.033978 28920 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:17:18.037369 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:18.019579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39540 (local address 127.23.200.254:33539)
0807 20:17:18.019614 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.019616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.020253 (+ 637us) server_negotiation.cc:407] Connection header received
0807 20:17:18.020523 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.020524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.020534 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.020562 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.021558 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.022259 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.024983 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.025312 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.025351 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.028281 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:18.028283 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:18.028283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:18.028290 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:18.031541 (+ 3251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:18.035476 (+ 3935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:18.035478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:18.035478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:18.035550 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:18.036469 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:18.036470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:18.036483 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:18.036616 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:18.036694 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:18.037189 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 20:17:18.037277 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:17:18.037930 28913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:18.019357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57280 (local address 127.23.200.252:33073)
0807 20:17:18.019396 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.019398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.019862 (+ 464us) server_negotiation.cc:407] Connection header received
0807 20:17:18.019952 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.019953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.019963 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.019991 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.023264 (+ 3273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.024008 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.025435 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.025757 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.025795 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.027954 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:18.027955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:18.027956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:18.027963 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:18.030867 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:18.035632 (+ 4765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:18.035633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:18.035634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:18.035698 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:18.036155 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:18.036156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:18.036157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:18.036302 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:18.036395 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:18.037752 (+ 1357us) server_negotiation.cc:299] Negotiation successful
0807 20:17:18.037843 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:17:18.040854 28919 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f6f97721102b45a98aaa102769b530e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } } }
I20260807 20:17:18.040956 28919 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:18.041149 28982 mvcc.cc:205] Tried to move back new op lower bound from 7316004200575987712 to 7316004200523685888. Current Snapshot: MvccSnapshot[applied={T|T < 7316004200575987712}]
I20260807 20:17:18.041527 28920 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f6f97721102b45a98aaa102769b530e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } } }
I20260807 20:17:18.041602 28920 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:17:18.043993 28787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } attempt: 3
Aug 07 20:17:18 dist-test-slave-2j7r krb5kdc[28527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.200.253@KRBTEST.COM for kudu/127.23.200.252@KRBTEST.COM
I20260807 20:17:18.049585 28971 catalog_manager.cc:1347] Generated new cluster ID: 1bf3d66647764bc2b5b1be60a78f7f9b
I20260807 20:17:18.049636 28971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:17:18.055902 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:18.044243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.200.252:33073 (local address 127.0.0.1:57286)
0807 20:17:18.044297 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 20:17:18.044309 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:17:18.044367 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:17:18.044540 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:17:18.044541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:17:18.044709 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:17:18.044755 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:18.044757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.045696 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.045697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:18.046526 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:17:18.046529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.047040 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.047042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:17:18.047104 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.047544 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:17:18.047549 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:17:18.049174 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:17:18.054667 (+ 5493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:18.054669 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:18.054669 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:18.054902 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:18.055160 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:17:18.055161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:17:18.055162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:17:18.055235 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:17:18.055565 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:17:18.055566 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:17:18.055631 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 20:17:18.055690 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 20:17:18.055800 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260807 20:17:18.056489 28913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:18.044332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57286 (local address 127.23.200.252:33073)
0807 20:17:18.044367 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.044369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.044377 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:17:18.044424 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.044425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.044435 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.044485 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.044834 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.045606 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.046609 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.046909 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.046983 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.049268 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:17:18.049269 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:17:18.049270 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:17:18.049277 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:17:18.054459 (+ 5182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:18.054999 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:18.055001 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:18.055001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:18.055079 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:17:18.055308 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:17:18.055309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:17:18.055309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:17:18.055465 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:17:18.055522 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:18.056304 (+ 782us) server_negotiation.cc:299] Negotiation successful
0807 20:17:18.056387 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 20:17:18.058370 28787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac: Bootstrap starting.
I20260807 20:17:18.059177 28787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac: Neither blocks nor log segments found. Creating new log.
I20260807 20:17:18.059494 28787 log.cc:824] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac: Log is configured to *not* fsync() on all Append() calls
I20260807 20:17:18.060788 28787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac: No bootstrap required, opened a new log
I20260807 20:17:18.061494 28971 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:17:18.062252 28971 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:17:18.062821 28787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } }
I20260807 20:17:18.062978 28787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a6d05f60d5462bb1fb63e5227e8bac, State: Initialized, Role: FOLLOWER
I20260807 20:17:18.063062 28787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } }
I20260807 20:17:18.063403 28787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:17:18.064324 28740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6f97721102b45a98aaa102769b530e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "31a6d05f60d5462bb1fb63e5227e8bac"
I20260807 20:17:18.064429 28740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:17:18.065223 28740 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6f97721102b45a98aaa102769b530e5 in term 1.
I20260807 20:17:18.065451 28739 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac [term 1 FOLLOWER]: Refusing update from remote peer f6f97721102b45a98aaa102769b530e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:17:18.066397 28958 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:17:18.068446 29011 mvcc.cc:205] Tried to move back new op lower bound from 7316004200692801536 to 7316004200523685888. Current Snapshot: MvccSnapshot[applied={T|T < 7316004200575987712}]
I20260807 20:17:18.063390 28992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } } }
I20260807 20:17:18.070008 28992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:17:18.070073 28992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6f97721102b45a98aaa102769b530e5. Latest consensus state: current_term: 1 leader_uuid: "f6f97721102b45a98aaa102769b530e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } } }
I20260807 20:17:18.070133 28992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:17:18.070164 28992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f6f97721102b45a98aaa102769b530e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } } }
I20260807 20:17:18.070207 28992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:17:18.070983 28988 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f6f97721102b45a98aaa102769b530e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f97721102b45a98aaa102769b530e5" member_type: VOTER last_known_addr { host: "127.23.200.254" port: 33539 } } peers { permanent_uuid: "31a6d05f60d5462bb1fb63e5227e8bac" member_type: VOTER last_known_addr { host: "127.23.200.253" port: 33859 } } peers { permanent_uuid: "c9535956ec4a4e578ce7b159188190e7" member_type: VOTER last_known_addr { host: "127.23.200.252" port: 33073 } } }
I20260807 20:17:18.071098 28988 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5 [sys.catalog]: This master's current role is: LEADER
I20260807 20:17:18.073158 28971 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6f97721102b45a98aaa102769b530e5: Generated new TSK 0
I20260807 20:17:18.073345 28971 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:17:18.073938 29008 catalog_manager.cc:1259] Loaded cluster ID: 1bf3d66647764bc2b5b1be60a78f7f9b
I20260807 20:17:18.074024 29008 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac: loading cluster ID for follower catalog manager: success
I20260807 20:17:18.075276 29008 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac: acquiring CA information for follower catalog manager: success
I20260807 20:17:18.075661 29008 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 31a6d05f60d5462bb1fb63e5227e8bac: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 20:17:18.099354 28792 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:18.097673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54202 (local address 127.23.200.253:33859)
0807 20:17:18.097718 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.097720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.098674 (+ 954us) server_negotiation.cc:407] Connection header received
0807 20:17:18.098778 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.098780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.098790 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.098820 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.099282 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54202: BlockingRecv error: recv got EOF from 127.0.0.1:54202 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 20:17:18.100236 28913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:18.098065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57292 (local address 127.23.200.252:33073)
0807 20:17:18.098108 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.098110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.099560 (+ 1450us) server_negotiation.cc:407] Connection header received
0807 20:17:18.099778 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.099779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.099790 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.099818 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.100173 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57292: BlockingRecv error: recv got EOF from 127.0.0.1:57292 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 20:17:18.100852 28663 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:18.098389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39550 (local address 127.23.200.254:33539)
0807 20:17:18.098428 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.098430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.099730 (+ 1300us) server_negotiation.cc:407] Connection header received
0807 20:17:18.100459 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.100460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.100470 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.100499 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.100789 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39550: BlockingRecv error: recv got EOF from 127.0.0.1:39550 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:17:18.105890 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:18.102386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54206 (local address 127.23.200.253:33859)
0807 20:17:18.102425 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.102427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.102646 (+ 219us) server_negotiation.cc:407] Connection header received
0807 20:17:18.102747 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.102748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.102788 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.102829 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:18.103043 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.103794 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.104888 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.105261 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.105302 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.105523 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:18.105648 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:18.105695 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:18.105799 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 20:17:18.105818 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 20:17:18.108959 28913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:18.108162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57304 (local address 127.23.200.252:33073)
0807 20:17:18.108196 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.108198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.108425 (+ 227us) server_negotiation.cc:407] Connection header received
0807 20:17:18.108527 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.108528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.108545 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.108574 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.108899 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57304: BlockingRecv error: recv got EOF from 127.0.0.1:57304 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:17:18.114274 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:18.111100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54220 (local address 127.23.200.253:33859)
0807 20:17:18.111131 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.111132 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.111380 (+ 248us) server_negotiation.cc:407] Connection header received
0807 20:17:18.111466 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.111467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.111476 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.111500 (+ 24us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:18.111650 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.112361 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.113505 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.113801 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.113838 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.114032 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:18.114065 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:18.114090 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:18.114183 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 20:17:18.114202 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 20:17:18.117293 28913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:18.116507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57308 (local address 127.23.200.252:33073)
0807 20:17:18.116544 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.116546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.116763 (+ 217us) server_negotiation.cc:407] Connection header received
0807 20:17:18.116854 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.116855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.116866 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.116894 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.117240 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57308: BlockingRecv error: recv got EOF from 127.0.0.1:57308 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:17:18.123461 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:18.120257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54228 (local address 127.23.200.253:33859)
0807 20:17:18.120290 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.120292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.120514 (+ 222us) server_negotiation.cc:407] Connection header received
0807 20:17:18.120604 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.120605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.120614 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.120638 (+ 24us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:18.120777 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.121611 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.122686 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.122985 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.123024 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.123204 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:18.123238 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:18.123274 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:18.123373 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 20:17:18.123390 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 20:17:18.126384 28913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:18.125607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57310 (local address 127.23.200.252:33073)
0807 20:17:18.125644 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.125646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.125857 (+ 211us) server_negotiation.cc:407] Connection header received
0807 20:17:18.125982 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.125983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.125994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.126024 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.126334 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57310: BlockingRecv error: recv got EOF from 127.0.0.1:57310 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:17:18.134754 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:18.131398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54244 (local address 127.23.200.253:33859)
0807 20:17:18.131432 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.131434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.131666 (+ 232us) server_negotiation.cc:407] Connection header received
0807 20:17:18.131756 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.131757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.131766 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.131792 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:18.131962 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.132689 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.133992 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.134288 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.134326 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.134509 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:18.134543 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:18.134569 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:18.134663 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 20:17:18.134681 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 20:17:18.137938 28913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:18.137156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57326 (local address 127.23.200.252:33073)
0807 20:17:18.137193 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.137195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.137430 (+ 235us) server_negotiation.cc:407] Connection header received
0807 20:17:18.137522 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.137524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.137536 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.137567 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.137871 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57326: BlockingRecv error: recv got EOF from 127.0.0.1:57326 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:17:18.150348 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:18.147080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54248 (local address 127.23.200.253:33859)
0807 20:17:18.147123 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.147126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.147383 (+ 257us) server_negotiation.cc:407] Connection header received
0807 20:17:18.147478 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.147479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.147491 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.147521 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:18.147683 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.148446 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.149568 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.149856 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.149894 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.150096 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:18.150131 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:18.150156 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:18.150251 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 20:17:18.150268 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 20:17:18.153573 28913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:18.152685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57342 (local address 127.23.200.252:33073)
0807 20:17:18.152723 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.152725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.152964 (+ 239us) server_negotiation.cc:407] Connection header received
0807 20:17:18.153059 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.153060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.153072 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.153148 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.153520 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57342: BlockingRecv error: recv got EOF from 127.0.0.1:57342 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:17:18.174312 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:18.170860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54258 (local address 127.23.200.253:33859)
0807 20:17:18.170917 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.170919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.171240 (+ 321us) server_negotiation.cc:407] Connection header received
0807 20:17:18.171350 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.171352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.171363 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.171392 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:18.171588 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.172321 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.173470 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.173748 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.173794 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.174015 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:18.174051 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:18.174078 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:18.174211 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 20:17:18.174231 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 20:17:18.177640 28913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:18.176694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57354 (local address 127.23.200.252:33073)
0807 20:17:18.176738 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.176740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.176994 (+ 254us) server_negotiation.cc:407] Connection header received
0807 20:17:18.177146 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.177147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.177160 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.177191 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.177583 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57354: BlockingRecv error: recv got EOF from 127.0.0.1:57354 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:17:18.213570 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:18.210983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54262 (local address 127.23.200.253:33859)
0807 20:17:18.211035 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.211037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.211143 (+ 106us) server_negotiation.cc:407] Connection header received
0807 20:17:18.211194 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.211195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.211207 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.211277 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:18.211392 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.212132 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.212852 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.213178 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.213244 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.213338 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:18.213372 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:18.213414 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:18.213469 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 20:17:18.213487 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 20:17:18.216104 28913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:18.215474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57362 (local address 127.23.200.252:33073)
0807 20:17:18.215516 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.215518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.215636 (+ 118us) server_negotiation.cc:407] Connection header received
0807 20:17:18.215685 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.215686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.215699 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.215746 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.216058 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57362: BlockingRecv error: recv got EOF from 127.0.0.1:57362 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:17:18.283164 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:18.281142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54274 (local address 127.23.200.253:33859)
0807 20:17:18.281212 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.281214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.281282 (+ 68us) server_negotiation.cc:407] Connection header received
0807 20:17:18.281336 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.281337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.281347 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.281391 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:18.281492 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.281983 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.282658 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.282829 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.282871 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.282972 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:18.282998 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:18.283031 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:18.283061 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 20:17:18.283082 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 20:17:18.285362 28913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:18.284758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57378 (local address 127.23.200.252:33073)
0807 20:17:18.284822 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.284824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.284849 (+ 25us) server_negotiation.cc:407] Connection header received
0807 20:17:18.284920 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.284921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.284933 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.284975 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.285310 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57378: BlockingRecv error: recv got EOF from 127.0.0.1:57378 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 20:17:18.417111 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:18.414633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54290 (local address 127.23.200.253:33859)
0807 20:17:18.414720 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.414724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.414817 (+ 93us) server_negotiation.cc:407] Connection header received
0807 20:17:18.414899 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.414900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.414910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.414955 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:18.415092 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.415594 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.416462 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.416652 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.416710 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.416841 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:18.416875 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:18.416915 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:18.416980 (+ 65us) server_negotiation.cc:299] Negotiation successful
0807 20:17:18.417000 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 20:17:18.419368 28913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:18.418779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57382 (local address 127.23.200.252:33073)
0807 20:17:18.418834 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.418836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.418875 (+ 39us) server_negotiation.cc:407] Connection header received
0807 20:17:18.418941 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.418942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.418954 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.419002 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.419316 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57382: BlockingRecv error: recv got EOF from 127.0.0.1:57382 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 20:17:18.679056 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:18.676763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54292 (local address 127.23.200.253:33859)
0807 20:17:18.676842 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.676844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.676867 (+ 23us) server_negotiation.cc:407] Connection header received
0807 20:17:18.676896 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.676897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.676906 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.676948 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:18.677112 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.677632 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.678441 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:18.678627 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:18.678681 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:18.678814 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:18.678845 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:18.678884 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:18.678953 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 20:17:18.678973 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 20:17:18.681331 28913 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:17:18.680811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57386 (local address 127.23.200.252:33073)
0807 20:17:18.680866 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:18.680868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:18.680877 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:17:18.680900 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:18.680901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:18.680911 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:18.680958 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:17:18.681273 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57386: BlockingRecv error: recv got EOF from 127.0.0.1:57386 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:17:19.006093 28954 catalog_manager.cc:1259] Loaded cluster ID: 1bf3d66647764bc2b5b1be60a78f7f9b
I20260807 20:17:19.006181 28954 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7: loading cluster ID for follower catalog manager: success
I20260807 20:17:19.007081 28954 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7: acquiring CA information for follower catalog manager: success
I20260807 20:17:19.007407 28954 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c9535956ec4a4e578ce7b159188190e7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 20:17:19.196710 29197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:19.194453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54302 (local address 127.23.200.253:33859)
0807 20:17:19.194581 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:19.194583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:19.194613 (+ 30us) server_negotiation.cc:407] Connection header received
0807 20:17:19.194684 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:19.194684 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:19.194696 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:19.194752 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:19.194850 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:19.195355 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:19.196113 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:19.196299 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:19.196345 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:19.196510 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:19.196538 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:19.196571 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:19.196611 (+ 40us) server_negotiation.cc:299] Negotiation successful
0807 20:17:19.196628 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260807 20:17:19.200764 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:19.198353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57394 (local address 127.23.200.252:33073)
0807 20:17:19.198512 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:19.198514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:19.198522 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:17:19.198548 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:19.198549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:19.198606 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:19.198671 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:19.198775 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:19.199270 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:19.199999 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:19.200201 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:19.200256 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:19.200413 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:19.200536 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:19.200607 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:19.200645 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 20:17:19.200669 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260807 20:17:19.205758 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:19.202662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.23.200.252:33073)
0807 20:17:19.202698 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:19.202699 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:19.202888 (+ 189us) server_negotiation.cc:407] Connection header received
0807 20:17:19.202952 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:19.202953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:19.202966 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:19.203013 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:19.203205 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:19.203930 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:19.204830 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:19.205034 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:19.205079 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:19.205344 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:19.205378 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:19.205431 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:19.205658 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 20:17:19.205684 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:17:19.205891 29197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:19.202512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54314 (local address 127.23.200.253:33859)
0807 20:17:19.202561 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:19.202563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:19.202716 (+ 153us) server_negotiation.cc:407] Connection header received
0807 20:17:19.202910 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:19.202911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:19.202925 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:19.202969 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:19.203098 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:19.203892 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:19.205058 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:19.205379 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:19.205442 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:19.205601 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:19.205631 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:19.205680 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:19.205784 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 20:17:19.205814 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:17:19.206396 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:17:19.202506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39556 (local address 127.23.200.254:33539)
0807 20:17:19.202658 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 20:17:19.202661 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:17:19.202690 (+ 29us) server_negotiation.cc:407] Connection header received
0807 20:17:19.202821 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:17:19.202822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:17:19.202881 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:17:19.203165 (+ 284us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:17:19.203211 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:19.203805 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:19.205119 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:17:19.205370 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:17:19.205438 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:17:19.205765 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:17:19.206072 (+ 307us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:17:19.206284 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 20:17:19.206302 (+ 18us) server_negotiation.cc:299] Negotiation successful
0807 20:17:19.206329 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
I20260807 20:17:19.207916 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 28547
I20260807 20:17:19.222245 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 28670
I20260807 20:17:19.239298 24355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskISIRft/build/release/bin/kudu with pid 28796
2026-08-07T20:17:19Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2991 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5954 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (277176 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST